XML Sitemap & Google News feeds
AdSense - Robot Denied (6 posts)

  1. prezla
    Posted 1 year ago #

    I got this information on AdSense:

    Our crawler was unable to access your page to determine its content and display relevant ads. When our crawler can’t access the content, often we won’t show ads resulting in lower revenue and coverage. At other times, we’ll show ads that are irrelevant resulting in lower CTR. Follow the links in the ‘How to fix.’ column to correct these errors and improve AdSense performance.

    Google suggests that I add this:

    User-agent: Mediapartners-Google

    I would like to do something like this:
    Search Engine Optimization for WordPress

    How to accomplish this, or maybe you have a better suggestion?
    My robots.txt


  2. RavanH
    Plugin Author

    Posted 1 year ago #

    Hi prezla, I see you already added the Mediapartners-Google rules... Do you still get the error?

  3. prezla
    Posted 1 year ago #

    Appears in a large number of false records to feed.
    How can I turn off the indexing feeds.

    Now I noticed that the generated files with the "/" at the end.


  4. RavanH
    Plugin Author

    Posted 1 year ago #

    That's a weird issue! The feed URL without the trailing slash seems to consist an XML Sitemap instead of the expected comment feed.

    It does not seem to come from my plugin (the output is different) but still... Can you test the following?

    1. Disable W3 Total Cache for a while and check if there is still this difference between http://srbin.info/2012/12/skandal-hrvati-miroslavljevo-jevandelje-je-nase-djelo-pisano-hrvatskom-cirilicom/feed and http://srbin.info/2012/12/skandal-hrvati-miroslavljevo-jevandelje-je-nase-djelo-pisano-hrvatskom-cirilicom/feed/ ? And if there is:
    2. disable the XML Sitemap & Google News Feeds plugin for a short time, and check again... then:
    3 disable all other plugins and finally switch to the default Twenty Thirteen theme for a second, until you find out when the difference is resolved.

  5. prezla
    Posted 1 year ago #

    It's a little harder.
    Since the site is at large visited on and off the cache may be a server crash ... I'll try early in the morning :).
    Thanks for your help
    in the meantime, I have set the parameters for this example

  6. RavanH
    Plugin Author

    Posted 1 year ago #

    Understandably you'd want to keep caching on. Is there an option in W3TC to disable caching of feeds? If so, that would be enough for the first step... And disabling XML Sitemap Feeds will not impact your visitors either but for the rest, you might want to put your site in maintenance mode using a plugin like "Ultimate Maintenance Mode" or others.

    Also make sure you are not using other plugins that create xml sitemap(s). Even in some SEO plugins there can be an option for xml sitemaps active...

Topic Closed

This topic has been closed to new replies.

About this Plugin

About this Topic