WordPress.org

Forums

WordPress SEO by Yoast
Sitemap Warning: Url blocked by robots.txt. (8 posts)

  1. fredel
    Member
    Posted 2 years ago #

    Hey,

    i get several warnings on sitemaps: my sitemap is http://bergtour-online.de/sitemap_index.xml and i got:

    webmaster tool:
    issue: Url blocked by robots.txt.
    description: Sitemap contains urls which are blocked by robots.txt.
    example: Sitemap: bergtour-online.de/post-sitemap.xml
    Value: http://bergtour-online.de/wp-includes/js/tinymce/plugins/wpgallery/img/t.gif

    has anyone an idea?

    thx all

    http://wordpress.org/extend/plugins/wordpress-seo/

  2. Eddie
    Member
    Posted 2 years ago #

    I'm having the same issue with one of my websites (working perfectly fine on others), for no apparent reason. The sitemap causing trouble is: http://belgrademyway.com/post-sitemap.xml

    It came to my mind that the image (1px) somehow got embedded in a post, but can't find it with Google, nor in the sitemap itself (image paths not displayed in the sitemap).

    Robots.txt itself isn't the problem, since it's identical to other WP based websites, hiding wp-admin and wp-includes folders from the bots.

    Anyone knows how to solve this issue?

  3. schnitzelboy
    Member
    Posted 2 years ago #

    I think I'm having the same issue as well.

    I'm getting a sitemap warning via google webmaster tools:
    Sitemap contains urls which are blocked by robots.txt.

    The offending URL is:
    http://www.thesocialwhat.com/wp-includes/js/tinymce/plugins/wordpress/img/trans.gif

    I've tried deactivating and deleting the plugin, and Google still claims the issue persists.

  4. Texiwill
    Member
    Posted 2 years ago #

    Hello,

    Check your robots.txt, this is normal for my site, I block certain locations from crawling as they are administrative and not for the general public. Most likely you have a link to the content somewhere else on your site.

    Best regards,
    Edward L. Haletky

  5. schnitzelboy
    Member
    Posted 2 years ago #

    Thanks for responding Edward -

    I agree it's robots.txt. Are you suggesting that within a post or page there's a reference to specific content that's somehow affecting/rewriting robots.txt?

  6. cmeinck
    Member
    Posted 1 year ago #

    I'm having the same issue. The dreaded robots.txt is blocking /wp-includes/js/tinymce/plugins/wordpress/img/trans.gif, which it should be blocking. When I click on the site map referenced in the error/warning, it's not there. From what I gather, it's somehow embedded on one of those pages? Am I correct?

    Has anyone successfully troubleshooted this issue? Any suggestions?

  7. cmeinck
    Member
    Posted 1 year ago #

    I wanted to follow up with a fix that others might find helpful. The trans.gif image was inserted in a page. Google Webmasters will show you the sitemap where the image exists, but it could be 1000 urls. I'm not sure, but I'm guessing you can configure Yoast to a smaller grouping of URL's. What I did was look at each URL that had 2 or more images and I would drop the link into the demo version of Screaming Spider, with the image URL window open. Most of these will come back as wp-content/... I went through about 60 or so URLs and then bingo. The culprit was a more tag with the trans.gif image at the end. Edited that page and removed the image.

    Hope this helps.

  8. Eddie
    Member
    Posted 1 year ago #

    I found a very simple way to fix my issue, probably could help others too. As cmeinck said, these images somehow got inserted into posts/pages.

    Go to your All posts, type t.gif (or trans.gif, or whatever image causes the problem) into the search field and find all the posts that have this image inserted (in my case it was just 2 of 100). Repeat the procedure for pages.

Topic Closed

This topic has been closed to new replies.

About this Plugin

  • WordPress SEO by Yoast
  • Frequently Asked Questions
  • Support Threads
  • Reviews

About this Topic

Tags

No tags yet.