search engine user agent 403

  • doug wilson

    (@dgswilson)


    12 years, 11 months ago

    Watching error logs, cpanel, seeing:

    [Mon May 30 05:51:31 2011] [error] [client 207.46.195.107] client denied by server configuration: /error.shtml
    [Mon May 30 05:51:31 2011] [error] [client 207.46.195.107] client denied by server configuration: /wpsite/2011

    And similarly:

    [Mon May 30 06:04:31 2011] [error] [client 207.46.204.192] client denied by server configuration: /error.shtml
    [Mon May 30 06:04:31 2011] [error] [client 207.46.204.192] client denied by server configuration: /wpsite/cap-n-trade

    I looked up msnbot IP range and this appears to be a legit IP. Not sure where the bot is seeing, for instance, wp/cap-n-trade instead of wp/cap-n-trade/… As far as I know all pages use the trailing slash. There are no settings denying bot from seeing pages.

    Maybe from another site linking to page? I’m wondering where I should look, I can’t think of anything else. Thanks, Doug

Viewing 5 replies - 1 through 5 (of 5 total)
  • Moderator Ipstenu (Mika Epstein)

    (@ipstenu)

    🏳️‍🌈 Advisor and Activist

    12 years, 11 months ago

    client denied by server configuration is not a WordPress thing, it’s quite literally your server 🙂 Mine has some fancy blocking in the .htaccess to stop people from getting in and throws that error all the time.

    Check your robots.txt and .htaccess to see if you’re blocking that IP/bot.

    Thread Starter doug wilson

    (@dgswilson)

    12 years, 11 months ago

    What is the rule that produces the error?

    Thread Starter doug wilson

    (@dgswilson)

    12 years, 11 months ago

    I’m using a whitelist in main htaccess. I just added a more specific “msnbot2.0b” for msnbot but I’m more puzzled by the “/” issue. That page, when I click on the link, has a trailing slash. There should be no non-/ url’s to not find.

    Moderator Ipstenu (Mika Epstein)

    (@ipstenu)

    🏳️‍🌈 Advisor and Activist

    12 years, 11 months ago

    There’s no one ‘rule’ that causes the error. I mean, I use this: http://perishablepress.com/5g-firewall-beta/

    Which as you can see is massive 😉 The whitelist fix should work.

    Don’t spend time fussing about the / – If you go to the link sans / it’ll put it in. That the bot’s coming with out the / could mean a lot of things, but none of them are hugely problematic.

    Thread Starter doug wilson

    (@dgswilson)

    12 years, 11 months ago

    Everything that happens on my site that I don’t understand is problematic. There’s a reason why everything happens and not knowing what the reasons are is a problem. But, I’ve always hated not knowing things. I’ll solve it, Thanks, Doug

Viewing 5 replies - 1 through 5 (of 5 total)
  • The topic ‘search engine user agent 403’ is closed to new replies.

Tags