Forum Replies Created

Viewing 15 replies - 31 through 45 (of 454 total)
  • I’ve checked and you are right. Does that have any impact on W3 total cache? Any settings I should be aware of?

    Have you any ideas about the missing page 1 issue? (it hasn’t re-ocurred).

    I’ve been having the same issue. After changing the Disc cache to basic, as instructed by and to allow the use of the plugin ‘Blackhole for Bad Bots’, we had an intermittent problem with the entire page 1 of the main front page not showing (ie the posts started at /page/2). Repeated cache clearing and resaving of permalinks initially cleared the problem, but after a while stopped working.
    I have now returned the setting to Disc: Advanced, and fingers crossed, it’s back to normal. I’ve disabled the bots plugin.

    If that’s the least that WP throws at you, you are very lucky. I’m not aware of any solution but I haven’t looked. Try Google! Or maybe ask the TinyMCE plugin support if you are using it.

    It’s just another of WordPress’ annoying habits. It will just throw them in randomly. And now you have to make 2 clicks to access the code editor as the direct link has unhelpfully been removed.

    I wonder if it’s to do with trying to block image hotlinking. I had this in htaccess for 24 hours (currently removed), which didn’t appear to be working according to various checker sites and inurl:carnethy.com -site:carnethy.com.

    # Prevent image hotlinking in WordPress
    RewriteEngine on
    RewriteCond %{HTTP_REFERER} !^$
    RewriteCond %{HTTP_REFERER} !^http://carnethy.com(/.*)?$ [NC]
    RewriteCond %{HTTP_REFERER} !^http(s)?://(www\.)?google.com(/.*)?$ [NC]
    RewriteCond %{HTTP_REFERER} !^http(s)?://(www\.)?facebook.com(/.*)?$ [NC]
    RewriteCond %{HTTP_REFERER} !^http(s)?://(www\.)?twitter.com(/.*)?$ [NC]
    RewriteCond %{HTTP_REFERER} !^http(s)?://(www\.)?yahoo.com(/.*)?$ [NC]
    RewriteCond %{HTTP_REFERER} !^http(s)?://(www\.)?bing.com(/.*)?$ [NC]
    RewriteCond %{REQUEST_URI}  !^/clansman-no-hotlink\.jpg$ [NC]
    RewriteRule \.([Jj][Pp][Gg]|[Jj][Pp][Ee][Gg]|gif|[Pp][Dd][Ff]|[Pp][Nn][Gg])$ - [F]  http://carnethy.com/clansman-no-hotlink.jpg [NC,R,L]

    Not sure if the syntax is correct.

    Interesting! I’ve also added quite a lot to robots.txt and added filters to Wordfence.

    I know this isn’t really relevant to your plugin but just for info –

    My main problem is still Bingbot. On our non-commercial running club site google bots are using about 1Gb a month, and Bingbots a massive 8Gb. Most of our referral traffic comes from Google, and a tiny amount from Bing. I don’t want to completely bar it but just make it use a reasonable amount.

    It appears to have 3 main IP addresses. I’ve more or less successfully throttled it (120 secs) which it seems to respond to by crawling in rapid batches with longer gaps. I’ve also added a lot of various disallows.

    The battle goes on!

    Thanks. That’s some list!

    I’m successfully blocking Amazon AWS using Wordfence. And I think I’ll make more use of its blocking facility.

    I’m having the same trouble. Awstats reports many many bots that should have been blocked are accessing the site and consuming bandwidth. A small example from the Bad Bots Table:

    Baiduspider Baidu Enabled Blocked 0
    Baiduspider-image Baiduspider-image Enabled Blocked 0
    Baiduspider-video Baiduspider-video/1.1 Enabled Blocked 0

    Awstats reports:
    Baiduspider Hits 792+9 Bandwidth 8.56 MB Last visit: 14 Jan 2019 – 11:11
    Baiduspider-image Hits 227 Bandwidth 15.65 MB 13 Jan 2019 – 16:34

    Bad Bots reports MJ12bot blocked 9710 times, but Awstats reports 1355 hits and 56Mb bandwidth.

    And so many others…

    The ones I’m thinking of are displaying normal viewing behaviour – a couple of visits to topical pages, not attacking behaviour.
    And yes, live traffic clearly shows the agent fields, and these ones don’t show any bot-like names.
    eg:

    Mozilla/5.0 (iPhone; CPU iPhone OS 12_1_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.0 Mobile/15E148 Safari/604.1
    Location: Edinburgh, United Kingdom

    digbymaass

    (@digbymaass)

    I ask because in the post https://wordpress.org/support/topic/you-have-been-banned-from-this-site/ the user had to whitelist themselves (although I know this should happen automatically for the admin who sets it up), in order to be able to preview posts.

    With quite a few admins/editors posting and previewing their posts is this then incorrect? Will they be able to preview posts no problem if their IP addresses are not whitelisted?

    digbymaass

    (@digbymaass)

    All 3 are correct and equivalent! The order and line spacing make no difference. Google “wordpress robots.txt” – there’s plenty of info online.

    Disallow: /wp-admin/
    Allow: /wp-admin/admin-ajax.php

    – is standard to prevent compliant bots from accessing /wp-admin/ but access to ajax is necessary for reasons I forget (and google wants it). You can look it up if you want to know the details. You still need to use robots.txt to control the behaviour of the ‘good’ bots, eg bingbot can eat up staggering amounts of bandwidth, at least it does on our site.

    That worked! I’m impressed! Can you explain it?

    • This reply was modified 11 months, 2 weeks ago by digbymaass.

    We don’t use Jetpack but we do use the plugin ‘Open Graph for Facebook, Google+ and Twitter Card Tags’. Don’t suppose you would know how to encode OG tags using this?

    Just don’t use the page scanner and it won’t happen. In any case, the overlords of WordPress have said that email encoders aren’t effective. Being a total sceptic I’m torn between not believing them and thinking anything’s better than nothing. At the moment we still use it, but it’s totally defeated by the OpenGraph plugin rendering the email addresses as normal text in the OG tags. Bah.

    Oh wait… I get it now!

Viewing 15 replies - 31 through 45 (of 454 total)