Support » Plugins » [Plugin: Badbehavior] User Agents to Whitelist Google Bots, Bing bots

  • I am using Badbehavior(BB) + Akismet and Cloudflare for my technology blog http://www.sparkwiz.com . Recently Google traffic started to drop and I finally found out it was because of bad behavior plugin blacklisting google bots. Also, when checked with google web masters, I found 500+ crawl errors(almost all the posts and pages). So, I have gone through the BBLogs and found user agents of Bots. Here are some few.

    Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
    Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.4 (KHTML, like Gecko; Google Web Preview) Chrome/22.0.1229 Safari/537.4
    FeedBurner/1.0 (http://www.FeedBurner.com)
    msnbot-media/1.1 (+http://search.msn.com/msnbot.htm)
    Mediapartners-Google
    SAMSUNG-SGH-E250/1.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 UP.Browser/6.2.3.3.c.1.101 (GUI) MMP/2.0 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)
    Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_1 like Mac OS X; en-us) AppleWebKit/532.9 (KHTML, like Gecko) Version/4.0.5 Mobile/8B117 Safari/6531.22.7 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)
    msnbot/2.0b (+http://search.msn.com/msnbot.htm)
    Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
    DoCoMo/2.0 N905i(c100;TB;W24H16) (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)

    You should add this to whitelist page of BB in WordPress one per line. Add this to Bad Behavior whitelist and update.

    Make a check in Google Webmaster and mostly, there will be a success in index, when you “Fetch as Google”.

    If you find some other user agents, please post here.

    Thanks in advance and I hope this list helps some one.

    Regards,
    Christopher

  • The topic ‘[Plugin: Badbehavior] User Agents to Whitelist Google Bots, Bing bots’ is closed to new replies.