Do search engines use Server Protocol 1.0 to crawl our website?

Viewing 3 replies - 1 through 3 (of 3 total)
  • Plugin Author AITpro

    (@aitpro)

    10 years, 7 months ago

    Everything that is legitimate these days will be using Server Protocol HTTP/1.1. Server Protocol HTTP/1.0 was phased out around 1997-1999.

    There are only 2 cases where I have seen something legitimate using HTTP/1.0.

    1. Verisign OCSP – uses both HTTP/1.1 and HTTP/1.0 (as a fallback I think), but this is a non-issue.
    1. Squid Proxy old versions – hackers/spammers/etc use the HTTP/1.0 older versions intentionally.

    Thread Starter icing

    (@icing)

    10 years, 7 months ago

    Thanks. So, from your reply, I can safely assume that Google crawler is using 1.1.

    Another thing that struck me was that even if a legitimate crawler is using 1.0, it doesn’t need to request wp-login.php. It can just crawl the rest of the pages. Right?

    Plugin Author AITpro

    (@aitpro)

    10 years, 7 months ago

    Everything that is legitmate uses HTTP/1.1.
    Anything that is using HTTP/1.0 is bad/malicious.

    Yes, since the intended purpose of the code is to protect the login page this code applies the security filter to only the login page.

Viewing 3 replies - 1 through 3 (of 3 total)
  • The topic ‘Do search engines use Server Protocol 1.0 to crawl our website?’ is closed to new replies.