WordPress.org

Ready to get started?Download WordPress

Forums

search engines, permissions, and robots.txt (4 posts)

  1. dm87
    Member
    Posted 9 years ago #

    This is the first time I have used php for a web site.

    First of all, I have set the permissions on the php files to 604, and things seem to be working as expected. Will this cause problems for search engine spiders, like googlebot?

    For search engines, what happens if I disallow access to the php files so that they will not be spidered. Does this prevent search engines from picking up the content? Is disallowing access to php files a good idea? A bad idea? Clearly I have no idea!

    Is there a good tutorial on this somewhere

  2. skippy
    Member
    Posted 9 years ago #

    If you want search engines to catalog your site, thereby including your site in search results, you should not block the search engine spiders. Your permissions should not cause a problem. If a human being can see them, so can googlebot.

  3. dnraikes
    Member
    Posted 8 years ago #

    Googlebot and some other spiders access my index.php page, but they never seem to crawl into the rest of my pages. Is there something special I should add to the robots.txt file or some special tags I should add to my wordpress header template?

    site:
    http://fitness.draikes.com

  4. whooami
    Member
    Posted 8 years ago #

    dnraikes, if googlebot knows where your site is, and it sounds like it does, it will eventually get to all of your other pages.

    Do not disallow access to .php files. There is no reason to, and it will keep googlebot and other spiders from moving around -- WP is built in PHP, your pages ALL end in .php :)

Topic Closed

This topic has been closed to new replies.

About this Topic