Title: robots.txt
Last modified: August 19, 2016

---

# robots.txt

 *  [rody](https://wordpress.org/support/users/rody/)
 * (@rody)
 * [15 years, 10 months ago](https://wordpress.org/support/topic/robotstxt-9/)
 * hi there
    im using robots.txt and i use this code
 * > User-agent: *
   >  Allow: / Disallow: /*?s= Disallow: /wp-admin/* Disallow: /wp-
   > content/* Disallow: /wp-includes/* Disallow: /wp-content/cache Disallow: /wp-
   > content/themes/* Disallow: /trackback Disallow: /comments Disallow: /category/
   > Disallow: */trackback Disallow: */comments Disallow: /*.js$ Disallow: /*.inc
   > $ Disallow: /*.css$ Disallow: /*.cgi$ Disallow: /*.html$ Disallow: /*.htm$ 
   > Disallow: /*.doc$ Disallow: /*.flv$ Disallow: /*.swf$ Disallow: /*.txt$ Disallow:/*.
   > rar$ Disallow: /*.zip$ Disallow: /page/* Disallow: /page/*/ Disallow: /pages/*
 * it is work or i must add more ?
 * i want just posts index in google

Viewing 5 replies - 1 through 5 (of 5 total)

 *  [webjunk](https://wordpress.org/support/users/webjunk/)
 * (@webjunk)
 * [15 years, 10 months ago](https://wordpress.org/support/topic/robotstxt-9/#post-1559768)
 * Why not remove the Allow in robots.txt then just put in the header of
    Posts.
   php: <meta name=”robots” content=”index,follow”> Pages.php <meta name=”robots”
   content=”noindex,nofollow”>
 *  Thread Starter [rody](https://wordpress.org/support/users/rody/)
 * (@rody)
 * [15 years, 10 months ago](https://wordpress.org/support/topic/robotstxt-9/#post-1559873)
 * can you be more specific
 *  [webjunk](https://wordpress.org/support/users/webjunk/)
 * (@webjunk)
 * [15 years, 10 months ago](https://wordpress.org/support/topic/robotstxt-9/#post-1559892)
 * Actually if you put in the Header.php for your theme in place of the original
   section for the robots enter this:
 * <?php if(is_single()) { ?>
    <meta name=”robots” content=”index,follow” /> <?php}
   else { <meta name=”robots” content=”noindex,nofollow” /> };?>
 * That should make the search engines only index and follow Single Post pages Everthing
   else will get ignored.
 *  Thread Starter [rody](https://wordpress.org/support/users/rody/)
 * (@rody)
 * [15 years, 10 months ago](https://wordpress.org/support/topic/robotstxt-9/#post-1559934)
 * oh
 * that is good
 * where i must add this code ?
 *  [webjunk](https://wordpress.org/support/users/webjunk/)
 * (@webjunk)
 * [15 years, 10 months ago](https://wordpress.org/support/topic/robotstxt-9/#post-1559976)
 * Header.php.

Viewing 5 replies - 1 through 5 (of 5 total)

The topic ‘robots.txt’ is closed to new replies.

 * In: [Fixing WordPress](https://wordpress.org/support/forum/how-to-and-troubleshooting/)
 * 5 replies
 * 2 participants
 * Last reply from: [webjunk](https://wordpress.org/support/users/webjunk/)
 * Last activity: [15 years, 10 months ago](https://wordpress.org/support/topic/robotstxt-9/#post-1559976)
 * Status: not resolved

## Topics

### Topics with no replies

### Non-support topics

### Resolved topics

### Unresolved topics

### All topics