Title: Robot.txt Question/Issue
Last modified: August 20, 2016

---

# Robot.txt Question/Issue

 *  [matthewjahn](https://wordpress.org/support/users/matthewjahn/)
 * (@matthewjahn)
 * [14 years, 6 months ago](https://wordpress.org/support/topic/robottxt-questionissue/)
 * Hey all,
 * So I’m new to robot.txt, and have a question;
 * I read that it’s good SEO to add the following code to your robot.txt file;
 *  User-agent: *
    Disallow: /wp-admin Disallow: /wp-includes Disallow: /wp-content/
   plugins Disallow: /wp-content/cache Disallow: /wp-content/themes Disallow: /trackback
   Disallow: /cgi-bin Disallow: /search Disallow: /feed Disallow: /rss Disallow:/
   comments/feed Disallow: /feed/$ Disallow: /*/feed/$ Disallow: /*/feed/rss/$ Disallow:/*/
   trackback/$
 * OK, so I didn’t really know how to do that, but luckily I found a place to enter
   this with the “SEO Ultimate” plugin. Easy, you just go to the File Editor section
   and enter the code into the correct field and it loads the code to your robot.
   txt file. I checked the file at [http://www.example.com/robot.txt](http://www.example.com/robot.txt)
   and all looked like it was supposed to.
 * Then I went to do the same thing on my blog. I have a separate WP installation
   on a subdirectory like [http://www.example.com/blog](http://www.example.com/blog).
   Now, I also read for this, the following code should be placed;
 *  User-agent: *
    Disallow: /blog/wp-admin Disallow: /blog/wp-includes Disallow:/
   blog/wp-content/plugins Disallow: /blog/wp-content/cache Disallow: /blog/wp-content/
   themes Disallow: /blog/trackback Disallow: /cgi-bin Disallow: /search Disallow:/
   blog/feed Disallow: /blog/rss Disallow: /blog/comments/feed Disallow: /blog/feed/
   $ Disallow: /blog/*/feed/$ Disallow: /blog/*/feed/rss/$ Disallow: /blog/*/trackback/
   $
 * Now, the problem is, when I do this on the blog, and try to look up my [http://www.example.com/blog/robot.txt](http://www.example.com/blog/robot.txt)
   file, I get a 404 error instead of the correct script.
 * Should I be putting the above code into the [http://www.example.com/robot.txt](http://www.example.com/robot.txt)
   file and not the subdirectory?
 * What am I doing wrong?
 * Any help would be great!
 * – Matthew

The topic ‘Robot.txt Question/Issue’ is closed to new replies.

## Tags

 * [robot.txt](https://wordpress.org/support/topic-tag/robot-txt/)

 * In: [Fixing WordPress](https://wordpress.org/support/forum/how-to-and-troubleshooting/)
 * 0 replies
 * 1 participant
 * Last reply from: [matthewjahn](https://wordpress.org/support/users/matthewjahn/)
 * Last activity: [14 years, 6 months ago](https://wordpress.org/support/topic/robottxt-questionissue/)
 * Status: not resolved

## Topics

### Topics with no replies

### Non-support topics

### Resolved topics

### Unresolved topics

### All topics