the useragent of that request, might very well be googlebot which seems to asynchronously request CSS or JS files. this thread discusses some more.
hope this helps,
frank
Thread Starter
gab81
(@gab81)
hi Frank,
I think the 410 gone solution is a good approach, is that difficult to do?
thanks,
Gab
Thread Starter
gab81
(@gab81)
hi Frank, sorry to write you again, can you share with me in PM (if there is here lol) how to enable the 410-gone solution?
thanks,
Gab
I would like to know, too 😉
Tom
@gab81 & @ecdltf
It’s not hard, but you need to have basic knowledge of htaccess. Here is the example code that need to added to your .htaccess file
# BEGIN 410 Gone
redirect 410 /wp-content/cache/autoptimize/js/autoptimize_xxxxxxxx.js
redirect 410 /wp-content/cache/autoptimize/js/autoptimize_12345678.js
# END 410 Gone
Change the “/wp-content/cache/…” with the url that causes 404 error. And make sure you have make a backup of the .htacess file. Because improper configuration in .htacess file can ruin your website and it’s SEO.
Thread Starter
gab81
(@gab81)
Hi there
I know .htaccess tho u dont say if its the root’s .htaccess or the wpcontent cache’s.. i think the root’s right??
That code requires manual input…..which would require going to check in the log for which .js are now 404s over time i guess…I have different .js files coming up as 404…..i was under the impression that there was a way for this to be dynamic and automatic?
Also i am thinking if google bot and company cannot crawl those .js from autoptimize, since they are -gone- a part from the 404, what is the loss in terms of functionality for the search engine? I mean would that hurt seo in your opinion? I work in seo but i am not great at javascript :).
I was reading an article the other day that gbot now understands more javascript > http://searchengineland.com/tested-googlebot-crawls-javascript-heres-learned-220157 so perhaps something we should be aware of moving forward.
Many thanks
Gabrio
So, I’m back from my holidays, glad you kids are getting along 😉
Put something like this (warning: untested) in wp-content/cache/autoptimize/.htaccess:
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule (.*) "-" [G,NC]
</IfModule>
If this works, you’ll probably want to update /wp-content/plugins/autoptimize/classes/autoptimizeCache.php to have that code as well (as autoptimizeCache.php writes .htaccess if not there).
The question of missing JS-files on SEO is a good one, but I have no answer except that in general Google needs to understand JS in order to be able to crawl JS-only sites built in e.g. AngularJS or Ember.
hope this helps,
frank
Thanks for the experimental .htaccess.
Currently I’m trying the other approach “making sure the files don’t get removed” as you proposed here.
I’ve moved out the AO cache folder from the regular cache folder, to avoid that everything gets deleted by WPSC.
Surely Google’s 404s will be avoided this way, but the problem remains that AO’s cache folder will grow ad infinitum.
So, I thought, I will let it grow, let’s say 4 weeks and than start to delete files older than 3 weeks by hand, once per month. This should still greatly reduce Google’s 404s, no? The rest would be covered by the .htaccess 410 entry as above.
But I’m not sure if removing files by hand will not disturb AO in some way?
removing files by hand won’t bother AO a bit 🙂
frank
I’m following this discussion because I’m getting lots of 404s from AO too. I’m using siteground supercacher plugin.
But I’m not a dev at all to tweak anything inside .htaccess if I’m not 100% sure is not going to break something else. :S
if I’m not 100% sure is not going to break something else. :S
I just copied Frank’s lines into my .htaccess. I’ll let you know if anything goes terribly wrong 🙂
Here you find some info on what is does.
Thread Starter
gab81
(@gab81)
hi Frank, I have just tested it and tried a direct hit from Google Cache and I get:
410
Gone
The requested resource
/wp-content/cache/autoptimize/js/autoptimize_xxxxxxxx2595f84404.js
is no longer available on this server and there is no forwarding address. Please remove all references to this resource.
so seems to be working 🙂
I did what you suggested and modded the .htaccess and the autoptimizeCache.php – there’s 2 instances there, right? it says write this code in the .htaccess ELSE write this code, etc.. so needs to be repeated twice, correct?
@davidroccato, is not hard at all, I suggest you to make a backup of the autoptimizeCache.php and the other .htaccess in the wp-content/cache/autoptimize/.htaccess folder.
Thread Starter
gab81
(@gab81)
btw I checked the 404 log of one of my plugins with 404 detection and no more 404s from AO, so that’s great….love the 410… GONE! 🙂
I did what you suggested and modded the .htaccess and the autoptimizeCache.php – there’s 2 instances there, right? it says write this code in the .htaccess ELSE write this code, etc.. so needs to be repeated twice, correct?
correct.
btw I checked the 404 log of one of my plugins with 404 detection and no more 404s from AO, so that’s great….love the 410… GONE! 🙂
I’m doing the happy-dance 😉
Keep on dancing!
Me, too, very happy 🙂
(Using the twofold approach here: AO’s cache moved out of reach of WPSC + rewrite to G)
