WordPress.org

Ready to get started?Download WordPress

Forums

Bot behavior (2 posts)

  1. mike
    Member
    Posted 9 years ago #

    The bots (particularly MSN and Google) are going nuts over my contact page(s). My stats indicate that they spend minutes at a time on a page that has little if any content. For instance, there is one case where the MSN bot reloaded this page 40 times in the space of 11 minutes. Very weird.
    My suspicion is that they are drawn to the page because where it says "posted by Michael" under each post, I have a unique url on 'michael' like contact/index.php?blog=name&title=name. So they keep going back there, cause I "tell" them to with my persistent linking.
    I doubt that it matters much, but it is interesting. Any ideas?

  2. wellard1981
    Member
    Posted 9 years ago #

    You could use a robots.txt in the root of your web site to stop this kind of activity.
    http://www.robotstxt.org/wc/robots.html

Topic Closed

This topic has been closed to new replies.

About this Topic

Tags

No tags yet.