• Resolved steve92

    (@steve92)


    Today I noticed someone had done 24 “captcha” searches and 2 “index/index” searches. In the admin setup, I only excluded the admin (myself), and they’re probably bots. How do I exclude bots?

Viewing 15 replies - 1 through 15 (of 16 total)
  • Plugin Author Mikko Saari

    (@msaari)

    It’s pretty hard, bot traffic is so invasive these days.

    You can use the relevanssi_bots_to_not_log filter hook to add bot user agents to block list, assuming these are bots that have an honest user agent string. Not all bots have. You can investigate your server access logs to find out who the bot is and to see if they can be blocked.

    Thread Starter steve92

    (@steve92)

    Relevanssi_log shows this:

    https://snipboard.io/4WsARo.jpg

    Plugin Author Mikko Saari

    (@msaari)

    Relevanssi logs don’t store the user agent, you need to go to the server access logs and find the requests for those queries there. Explaining all that in detail goes beyond the support we can provide here; I’m sure there are lots of good sources for that information elsewhere.

    Thread Starter steve92

    (@steve92)

    What I mean is this: my site doesn’t require registration to visit it. I’d like the search statistics by Relevanssi to show only user searches, not bots. The Altervista server access logs show total access (bots, users, etc.).

    Thread Starter steve92

    (@steve92)

    This code doesn’t work, it shows error

    add_filter( 'relevanssi_bots_to_not_log', function( $bots ) {
    return array_merge( $bots, array( 'Bad Bot' => 'BadBot' ) );
    }

    Plugin Author Mikko Saari

    (@msaari)

    Yes, I know, and that’s what I’m trying to help you with. You need to look at the server access logs to find out which bot is accessing your site, then you can use the filter function to add the user agent to the block list.

    The sample code is missing a bit. It should look like this:

    add_filter( 'relevanssi_bots_to_not_log', function( $bots ) {
    return array_merge( $bots, array( 'Bad Bot' => 'BadBot' ) );
    } );
    Thread Starter steve92

    (@steve92)

    I put them all, maybe someone is not necessary.

    add_filter( 'relevanssi_bots_to_not_log', function( $bots ) {
    return array_merge( $bots,
    array(
    'Google Mediapartners' => 'Mediapartners-Google',
    'GoogleBot' => 'Googlebot',
    'Bing' => 'Bingbot',
    'Yahoo' => 'Slurp',
    'DuckDuckGo' => 'DuckDuckBot',
    'Baidu' => 'Baiduspider',
    'Yandex' => 'YandexBot',
    'Sogou' => 'Sogou',
    'Exalead' => 'Exabot',
    'Majestic' => 'MJ12Bot',
    'Perplexity' => 'PerplexityBot',
    'ByteDance' => 'Bytespider') );
    } );
    • This reply was modified 1 day, 15 hours ago by steve92.
    Plugin Author Mikko Saari

    (@msaari)

    A) Oops, I forgot this was changed and the documentation has expired. The filter hook is now called relevanssi_bots_to_block. The functionality is otherwise the same.

    B) There’s no need to list the default bots, they’re already included.

    Thread Starter steve92

    (@steve92)

    is this okay?

    add_filter( 'relevanssi_bots_to_block', function( $bots ) {
    return array_merge( $bots, array( 'Bad Bot' => 'BadBot' ) );
    } );

    • This reply was modified 1 day, 15 hours ago by steve92.
    Plugin Author Mikko Saari

    (@msaari)

    Yes, that will add the bot to the block list.

    Thread Starter steve92

    (@steve92)

    You wrote earlier that the list in the array isn’t needed. Does this code block all bots?

    Plugin Author Mikko Saari

    (@msaari)

    It adds “Bad Bot” to the existing list of blocked bots.

    Thread Starter steve92

    (@steve92)

    I can’t figure it out: what is Bad bot? Should I just use this?

    Plugin Author Mikko Saari

    (@msaari)

    That’s what you need the access log for. The access log has every request made on your server, so you find the unwanted searches there and then see what the user agent for that search is, and add it to the block list.

    Here’s a guide for the topic.

    Thread Starter steve92

    (@steve92)

    If I put this code it includes them all and I’ve solved the problem.

    add_filter( 'relevanssi_bots_to_not_log', function( $bots ) {
    return array_merge( $bots,
    array(
    'Google Mediapartners' => 'Mediapartners-Google',
    'GoogleBot' => 'Googlebot',
    'Bing' => 'Bingbot',
    'Yahoo' => 'Slurp',
    'DuckDuckGo' => 'DuckDuckBot',
    'Baidu' => 'Baiduspider',
    'Yandex' => 'YandexBot',
    'Sogou' => 'Sogou',
    'Exalead' => 'Exabot',
    'Majestic' => 'MJ12Bot',
    'Perplexity' => 'PerplexityBot',
    'ByteDance' => 'Bytespider') );
    } );
    • This reply was modified 1 day, 1 hour ago by steve92.
Viewing 15 replies - 1 through 15 (of 16 total)

You must be logged in to reply to this topic.