Title: Robots.txt blocking Google
Last modified: August 22, 2016

---

# Robots.txt blocking Google

 *  [atolmaproductions](https://wordpress.org/support/users/atolmaproductions/)
 * (@atolmaproductions)
 * [11 years, 4 months ago](https://wordpress.org/support/topic/robotstxt-blocking-google-2/)
 * I have been told by Google News that parts of [my site](http://themusicuniverse.com)
   are being blocked from their crawlers, therefore, they can’t include my site 
   in Google News until this is corrected. I’ve done everything I can think of to
   fix it, even deactiving the WordPress SEO plugin and installing Yoast SEO plugin.
   The latter I heard is easier to deal with.
 * No matter what I do, stuff seems to be blocked from the crawlers and some it 
   seems to be Google AdSense and Javascript files. Here’s a rundown of what’s blocked.
 * [http://pagead2.googlesyndication.com/pagead/show_ads.js](http://pagead2.googlesyndication.com/pagead/show_ads.js)
   Script Blocked
    [http://www.tqlkg.com/image-7636937-10386908](http://www.tqlkg.com/image-7636937-10386908)
   Image Blocked [http://www.lduhtrp.net/image-7636937-10675279](http://www.lduhtrp.net/image-7636937-10675279)
   Image Blocked [https://scontent-a.xx.fbcdn.net/hprofile-xfa1/v/l/t1.0-1/p50x50/10805808_10154898303465416_8412066579935943526_n.jpg?oh=d6f0a147a5f2d9a853b57932bd79d20c&oe=55032BE5](https://scontent-a.xx.fbcdn.net/hprofile-xfa1/v/l/t1.0-1/p50x50/10805808_10154898303465416_8412066579935943526_n.jpg?oh=d6f0a147a5f2d9a853b57932bd79d20c&oe=55032BE5)
   Image Blocked [https://cdn.syndication.twimg.com/widgets/timelines/358011796092772352?domain=themusicuniverse.com&lang=en&t=1577185&callback=twttr.tfw.callbacks.tl_358011796092772352_1&suppress_response_codes=true](https://cdn.syndication.twimg.com/widgets/timelines/358011796092772352?domain=themusicuniverse.com&lang=en&t=1577185&callback=twttr.tfw.callbacks.tl_358011796092772352_1&suppress_response_codes=true)
   Script Blocked [https://public-api.wordpress.com/wp-admin/rest-proxy/](https://public-api.wordpress.com/wp-admin/rest-proxy/)
 * My robots.txt file looks like this:
 * User-agent: *
    Disallow: /wp-admin/
 * Sitemap: [http://themusicuniverse.com/sitemap_index.xml](http://themusicuniverse.com/sitemap_index.xml)
   
   Sitemap: [http://themusicuniverse.com/sitemap.xml](http://themusicuniverse.com/sitemap.xml)
 * Any help to fix this would be greatly appreciated!
 * Thanks,
    Buddy

Viewing 1 replies (of 1 total)

 *  [marktmccabe](https://wordpress.org/support/users/marktmccabe/)
 * (@marktmccabe)
 * [11 years, 4 months ago](https://wordpress.org/support/topic/robotstxt-blocking-google-2/#post-5610173)
 * TRy installing this plugin it will work for some but not all.
 * Async JS and CSS
 * I am struggling with Rest-Proxy blocking my crawlers – anyone help with that?
 * Kind Regards
 * Mark
    [](http://website-development.org)

Viewing 1 replies (of 1 total)

The topic ‘Robots.txt blocking Google’ is closed to new replies.

## Tags

 * [crawler](https://wordpress.org/support/topic-tag/crawler/)
 * [robots](https://wordpress.org/support/topic-tag/robots/)

 * 1 reply
 * 2 participants
 * Last reply from: [marktmccabe](https://wordpress.org/support/users/marktmccabe/)
 * Last activity: [11 years, 4 months ago](https://wordpress.org/support/topic/robotstxt-blocking-google-2/#post-5610173)
 * Status: not resolved

## Topics

### Topics with no replies

### Non-support topics

### Resolved topics

### Unresolved topics

### All topics