Strange research
-
Today I noticed someone had done 24 “captcha” searches and 2 “index/index” searches. In the admin setup, I only excluded the admin (myself), and they’re probably bots. How do I exclude bots?
-
It’s pretty hard, bot traffic is so invasive these days.
You can use the
relevanssi_bots_to_not_logfilter hook to add bot user agents to block list, assuming these are bots that have an honest user agent string. Not all bots have. You can investigate your server access logs to find out who the bot is and to see if they can be blocked.Relevanssi_log shows this:
Relevanssi logs don’t store the user agent, you need to go to the server access logs and find the requests for those queries there. Explaining all that in detail goes beyond the support we can provide here; I’m sure there are lots of good sources for that information elsewhere.
What I mean is this: my site doesn’t require registration to visit it. I’d like the search statistics by Relevanssi to show only user searches, not bots. The Altervista server access logs show total access (bots, users, etc.).
This code doesn’t work, it shows error
add_filter( 'relevanssi_bots_to_not_log', function( $bots ) {
return array_merge( $bots, array( 'Bad Bot' => 'BadBot' ) );
}Yes, I know, and that’s what I’m trying to help you with. You need to look at the server access logs to find out which bot is accessing your site, then you can use the filter function to add the user agent to the block list.
The sample code is missing a bit. It should look like this:
add_filter( 'relevanssi_bots_to_not_log', function( $bots ) {
return array_merge( $bots, array( 'Bad Bot' => 'BadBot' ) );
} );I put them all, maybe someone is not necessary.
add_filter( 'relevanssi_bots_to_not_log', function( $bots ) {
return array_merge( $bots,
array(
'Google Mediapartners' => 'Mediapartners-Google',
'GoogleBot' => 'Googlebot',
'Bing' => 'Bingbot',
'Yahoo' => 'Slurp',
'DuckDuckGo' => 'DuckDuckBot',
'Baidu' => 'Baiduspider',
'Yandex' => 'YandexBot',
'Sogou' => 'Sogou',
'Exalead' => 'Exabot',
'Majestic' => 'MJ12Bot',
'Perplexity' => 'PerplexityBot',
'ByteDance' => 'Bytespider') );
} );-
This reply was modified 1 day, 15 hours ago by
steve92.
A) Oops, I forgot this was changed and the documentation has expired. The filter hook is now called
relevanssi_bots_to_block. The functionality is otherwise the same.B) There’s no need to list the default bots, they’re already included.
is this okay?
add_filter( 'relevanssi_bots_to_block', function( $bots ) {
return array_merge( $bots, array( 'Bad Bot' => 'BadBot' ) );
} );-
This reply was modified 1 day, 15 hours ago by
steve92.
Yes, that will add the bot to the block list.
You wrote earlier that the list in the array isn’t needed. Does this code block all bots?
It adds “Bad Bot” to the existing list of blocked bots.
I can’t figure it out: what is Bad bot? Should I just use this?
That’s what you need the access log for. The access log has every request made on your server, so you find the unwanted searches there and then see what the user agent for that search is, and add it to the block list.
If I put this code it includes them all and I’ve solved the problem.
add_filter( 'relevanssi_bots_to_not_log', function( $bots ) {
return array_merge( $bots,
array(
'Google Mediapartners' => 'Mediapartners-Google',
'GoogleBot' => 'Googlebot',
'Bing' => 'Bingbot',
'Yahoo' => 'Slurp',
'DuckDuckGo' => 'DuckDuckBot',
'Baidu' => 'Baiduspider',
'Yandex' => 'YandexBot',
'Sogou' => 'Sogou',
'Exalead' => 'Exabot',
'Majestic' => 'MJ12Bot',
'Perplexity' => 'PerplexityBot',
'ByteDance' => 'Bytespider') );
} );-
This reply was modified 1 day, 1 hour ago by
steve92.
-
This reply was modified 1 day, 15 hours ago by
You must be logged in to reply to this topic.