• Hi,

    I don’t know why, but I noticed that bots from google, bing and others are crawling weird, nonexistent, urls on my site.

    My site is a multisite and has a couple different domain names. The urls the bots are crawling looks like this:

    http://www.domain_name_one.com/domain_name_two.com/random-phases

    So these are the random phases I have been getting:
    newborn-cub
    radwan-ziadeh
    baju-songket-putih
    norman-smitherman
    and hundreds more coming in every minutes or so

    Of course, these urls don’t exist and in fact, the second domain name is an old site that I already deleted from my multisite. There are no links between any of the sites I have so there shouldn’t be any reason why it would even reference the two sites together.

    I have been hacked a couple of times the past few months and I am not sure if this is related but I want to know if I should try fixing this and if so, how I can fix it.

    i tried using robots.txt to block the bots access but I haven’t seen any improvements.

    I used the following:

    Disallow: /domain_name_two.com/*

    Thanks in advance.

The topic ‘Bots Crawling Nonexistent URLs!’ is closed to new replies.