Does wordpress notify crawlers and robots of the sites installation? (2 posts)

  1. rtaft
    Posted 9 years ago #

    I ask this because withing a day of me installing wordpress on my home system, it was hit by googlebot. I installed this on a private system, private address, that at the time had never been crawled (that I know of). After doing a second install Sunday, into a second directory, that was crawled on monday. I have many subdirectories on my server, all protected by anonymity. When you go to my site, there is just the Fedora Core test page. There are no links to browse to other directories on the server. Yet BOTH of my wordpress installs have been crawled, and NOTHING else on my server. I understand that I can use a robots.txt file to correct this, assuming the crawlers obey.

  2. drmike
    Posted 9 years ago #

    Are you sending out pings though? Dashboard -> Options -> I believe it's the General subpage at the bottom -> Update Services. If so, that's probably how they found you.

Topic Closed

This topic has been closed to new replies.

About this Topic


No tags yet.