Blocked Microsoft Hits

  • Resolved wolfkettler

    (@wolfkettler)


    7 years, 9 months ago

    Today I had in excess of 16,000 (!) hits from a number of Microsoft IPs:
    40.77.167.44, 40.77.167.76, 40.77.167.48, 157.55.39.176, 157.55.39.231, 207.46.13.116, 157.55.39.167, 40.77.167.137.

    Apart from the fact that 16,000 hits/requests is a large number for a small website, all the requests were made for non-existent PDF files with a vaguely medical theme, such as https:…/yasmin-antibabypille-kaufen.pdf,
    https:…/cialis-price-in-south-africa.pdf
    https:…/can-i-buy-propecia-over-the-counter-in-canada.pdf
    https:…/crestor-20-mg-90-tablet.pdf

    These are very specific file names, they are not related to the topic of my site at all and none have ever existed on my website.

    I have spoken to my hosting provider and they think that I have nothing to worry about. I still find the episode very, very strange and was hoping to find out more.

    I am going to contact Microsoft to request more information but I do not really expect to hear back from them.

    Thanks,
    Wolf

Viewing 11 replies - 1 through 11 (of 11 total)
  • Stratosphere

    (@stratosphere)

    7 years, 9 months ago

    If it’s any consolation, I have a similar problem with MSNbots, sending bad links. I discussed with them, sent them a list, they did nothing. So, since I don’t need their traffic (Google is primary search engine) I block MSNbots. It seems they are too busy making billions to bother about website owners.

    mountainguy2

    (@mountainguy2)

    7 years, 9 months ago

    The next great frontier for Wordfence is to help us get control of the search indexing bots, especially to help us figure out when they’re fake. I’ve spent hours and hours trying to figure out how to reduce bandwidth taken by these idiots and have not have much success. And yes, the corporate culture behind the legitimate search indexing bots (spiders) is such that they expect us to pay our own hard earned money for server resources they can use with impunity. MTN

    Thread Starter wolfkettler

    (@wolfkettler)

    7 years, 9 months ago

    Interesting. What confuses me is how/why they try to access URLs that have never existed on my site. The ‘links’ are complete fantasy! Why would that happen and what would anybody gain from it?

    Stratosphere

    (@stratosphere)

    7 years, 9 months ago

    Thanks Mountainguy, yes it is the next frontier to tackle search bots that are somehow rigged and are faked.
    As said, I sent msn/bing all the info on bad links coming from specific sites, that I suspect are sending the bad traffic. they expect me to wait until their bots hit the correct pages/posts on my site even though they have a clear sitemap. It’s been four months now I finally told them to follow Google’s search bots and just copy them which are accurate.

    Wolf: in my case whomever is using msnbots have somehow engineered fake links (meaning the url’s do not exist on my site) and I am no expert but its done by scripting the link.
    Example: http://mysite.com/BadLinkhere/
    They add the bad link at the end of the url.

    There is very little I can do to stop it and msn/bing refuse to check these links and find where they are coming from or disavow them. It’s very frustrating, I don’t have the time to investigate nor a tech team to help.
    Yes, they are “fantasy links” that do not exist on your site or mine.
    What do they gain? I do believe they send traffic to their spammy sites this way, using our websites but don’t know how to prevent it.
    I have traced a few of these sites and some are non-existent domains, meaning they don’t have a website, they are just a domain name.

    I have used robots.txt to ban msnbots and yandex bots, so far only yandex is adhering to it.

    wfyann

    (@wfyann)

    7 years, 9 months ago

    Hi @wolfkettler,

    One thing you can do to reduce the impact on your resources is to set rate limiting rules.

    To do so:

    • Go to the Wordfence Firewall page
    • Click the Rate Limiting tab
    • Hit the Save Options button once you’re done
    Stratosphere

    (@stratosphere)

    7 years, 9 months ago

    Tried rate limiting, they still get through, the only thing that worked is banning msnbots and their IP ranges. But I still see them being blocked every day, approximately 10 a day or more.

    Thread Starter wolfkettler

    (@wolfkettler)

    7 years, 9 months ago

    @stratosphere, when you say “They add the bad link at the end of the url” – do you know how this works?

    Wolf

    Thread Starter wolfkettler

    (@wolfkettler)

    7 years, 9 months ago

    @wfyann, thank you for this. Word Fence is blocking traffic very efficiently, although the same fantasy URLs are being crawled now by Google, too.

    What I am really worried about is what this may do to my reputation.

    I am also very curious how – practically – someone may profit from crawling my site for non-existent pages.
    I have a real problem with things that I do not understand 🙂

    Wolf

    wfyann

    (@wfyann)

    7 years, 8 months ago

    Hi @wolfkettler,

    I found this article on the Google documentation platform which discusses crawling related issues.

    Thread Starter wolfkettler

    (@wolfkettler)

    7 years, 8 months ago

    That’s very interesting. Thank you very much.
    Although, I still don’t understand how anyone would benefit from it.

    Thanks again!

    wfyann

    (@wfyann)

    7 years, 8 months ago

    Hi @wolfkettler,

    What you could be dealing here are fake crawlers. The ultimate goal of such method is to steal content.

    You could try and block fake Google crawlers but please note that this option may have some downsides

Viewing 11 replies - 1 through 11 (of 11 total)
  • The topic ‘Blocked Microsoft Hits’ is closed to new replies.