WordPress.org

Ready to get started?Download WordPress

Forums

Is this the Best Robots.txt Example? (4 posts)

  1. totalgaz
    Member
    Posted 3 years ago #

    Using Google webmaster tools I realized my robots.txt file was not optimal for crawling and indexing the blog.

    I cam across what WordPress says is a good example of a SEO robots.txt file. I was wondering if you think this is the best info for an average WordPress blog to have in its robots.txt file.

  2. Samuel Wood (Otto)
    Tech Ninja
    Posted 3 years ago #

    If you use WordPress in the root of your site, it auto-generates a robots.txt output for you, based on the privacy settings. I tend to just leave it at that.

    You can, of course, make your own based on your own site, but I think most people just want their whole site to be indexed.

  3. totalgaz
    Member
    Posted 3 years ago #

    Hay Otto,

    So the example WordPress gave in their codex page is it the same as it does automatically for a standard WordPress blog?

  4. Samuel Wood (Otto)
    Tech Ninja
    Posted 3 years ago #

    No, the example is just an example. It's not a recommended example or something like that.

Topic Closed

This topic has been closed to new replies.

About this Topic