Images, CSS & JS get blocked for Googlebot by Jetpack

  • Resolved MasterBen

    (@masterben)


    8 years, 8 months ago

    Hello,

    Please be advised that this question pertains to a blog of mine which features pornographic material (NSFW!)

    As you may have heard, Google has been sending emails to webmasters in which they insist that all images, CSS and javascript on your site must be accessible (not blocked) for Googlebot. One can check if that is the case in Google Webmaster Tools (Search Console) by pressing “Fetch and Render” option.

    Now, when my Jetpack plugin is turned off, I always get “complete” success with “Fetch and render” in Google WMT. However, as soon as I activate Jetpack, I can’t get “complete” success with “fetch and render” but only “partial”. When I click that particular URL that has been fetched in WMT, I get something like this:

    http://www.roughstraightmen.com/wp-content/misc/fetchandrender1.jpg
    http://www.roughstraightmen.com/wp-content/misc/fetchandrender2.jpg
    http://www.roughstraightmen.com/wp-content/misc/fetchandrender3.jpg

    You can see that the images, css and JS files that are being blocked are pretty much arbitrary… even when the same post on the blog is tested at different times, different files appear to be blocked for Googlebot.

    This is a serious bug and drawback for webmasters, because Google has announced that if a site has css, js and image files that are inaccessible for its googlebot, they will penalize that site accordingly in their search results.

    I only use these two modules in Jetpack: wordpress.com stats and sharing. I also use the plugin called “Popular Posts Tabbed Widget for Jetpack” which relies on Jetpack’s stats to display popular posts. I like these modules and would like to keep them on my blog, but not at the expense of Site’s google ranking.

    Does anyone know how this blocking of files can be fixed?

    https://wordpress.org/plugins/jetpack/

Viewing 4 replies - 1 through 4 (of 4 total)
  • Plugin Author Jeremy Herve

    (@jeherve)

    Jetpack Mechanic 🚀

    8 years, 8 months ago

    Jetpack doesn’t block access to images, CSS, and Javascript files on your site. It seems that your site is set up to block Google to index quite a few directories, as you can see here:
    http://www.roughstraightmen.com/robots.txt

    That’s not recommended, as you can read here:
    https://yoast.com/dont-block-your-css-and-js-files/

    You can find out how to fix this issue here:
    http://www.shoutmeloud.com/fix-googlebot-cannot-access-css-js-warning.html

    Thread Starter MasterBen

    (@masterben)

    8 years, 8 months ago

    I’ve deleted all the items with disallow in my robots.txt file now. There are only several allow items in my robots.txt file now.

    However, when I turn on Jetpack, I still get “Partial” success, whereas without Jetpack I usually get “complete” success with fetch and render…

    Admittedly, I do get “partial” even with Jetpack turned off SOMETIMES, but when Jetpack is turned on I ALWAYS get “partial”.

    I know that Jetpack isn’t supposed to block googlebot, but it seems it does for some odd reason.

    Thread Starter MasterBen

    (@masterben)

    8 years, 8 months ago

    By the way, those items in my robots.txt file which had the disallow command, they pertain to nonexistent folders (such as /recommends and /promovideos). Those folders don’t actually exist, they were there only to prevent googlebot from indexing my jump script which uses such syntax. As far as I know, they shouldn’t have blocked googlebot (or anything else for that matter).

    However, even those items have been deleted so my robots.txt file shouldn’t block anything anymore.

    Plugin Author Jeremy Herve

    (@jeherve)

    Jetpack Mechanic 🚀

    8 years, 8 months ago

    when I turn on Jetpack, I still get “Partial” success, whereas without Jetpack I usually get “complete” success with fetch and render…

    That’s to be expected, as Jetpack will add some resources that can’t be crawled, like the scripts used to calculate sharing buttons for the sharing buttons. That shouldn’t hurt your rankings though. As long as the resources are not images and pages that you want Google to index, you can ignore them.

Viewing 4 replies - 1 through 4 (of 4 total)
  • The topic ‘Images, CSS & JS get blocked for Googlebot by Jetpack’ is closed to new replies.