possible to disallow archive.org ?

  • Resolved hai3009

    (@hai3009)


    8 years, 2 months ago

    Hi,

    is it possible to dissalow the archive.org spider?
    Reason is archieve.org did not longer respect the robots.txt entrys.

    This string is inside the logfile:
    "Mozilla/5.0 (compatible; archive.org_bot; Wayback Machine Live Record; +http://archive.org/details/archive.org_bot)"

    Any help or Idea for blocking this?
    Kind regards!

    • This topic was modified 8 years, 2 months ago by hai3009.
    • This topic was modified 8 years, 2 months ago by hai3009.
Viewing 2 replies - 1 through 2 (of 2 total)
Viewing 2 replies - 1 through 2 (of 2 total)

The topic ‘possible to disallow archive.org ?’ is closed to new replies.