WordPress.org

Forums

Is this the Best Robots.txt Example? (4 posts)

  1. totalgaz
    Member
    Posted 4 years ago #

    Using Google webmaster tools I realized my robots.txt file was not optimal for crawling and indexing the blog.

    I cam across what WordPress says is a good example of a SEO robots.txt file. I was wondering if you think this is the best info for an average WordPress blog to have in its robots.txt file.

  2. If you use WordPress in the root of your site, it auto-generates a robots.txt output for you, based on the privacy settings. I tend to just leave it at that.

    You can, of course, make your own based on your own site, but I think most people just want their whole site to be indexed.

  3. totalgaz
    Member
    Posted 4 years ago #

    Hay Otto,

    So the example WordPress gave in their codex page is it the same as it does automatically for a standard WordPress blog?

  4. No, the example is just an example. It's not a recommended example or something like that.

Topic Closed

This topic has been closed to new replies.

About this Topic