Support » Fixing WordPress » robots.txt with WordPress

  • i am trying to figure out just what i should disallow in the robots.txt file in my WordPress directory.. i want my site and it’s content indexed so i want to make sure i do now disallow andything i shouldn’t..

    thx

Viewing 4 replies - 1 through 4 (of 4 total)
  • are there any files or directories i should be disallowing?

    You might disallow the wp-admin and wp-includes directory?

    There’s no right answer, it’s up to you entirely. If there’s anything you don’t want the search engine going to, disallow it. It doesn’t hurt to be safe 🙂

    appreciate it! that is exactly what i was going to do. i wasn’t sure if there was anything in those directories i would want spidered. thanks for the help.

    To be honest, those directories probably wouldn’t get spidered anyway, there are no links to them from anywhere, unless you count the stylesheet link — but then that’s to wp-content anyway.

    But it’s your call, it won’t damamge the performance of your site or anything.

Viewing 4 replies - 1 through 4 (of 4 total)
  • The topic ‘robots.txt with WordPress’ is closed to new replies.