Podz: robots.txt.
You can put something like this in robots.txt:
User-agent: googlebot
Disallow: /mypornfiles
googlebot will not index /mypornfiles If you add that to robots.txt. 🙂
To block all bots (Not just google), Make the user agent this:
User-agent: *
That’s all. robots.txt goes inside the root folder, and that’s it. For multiple files, Just keep adding the disallow code (ex:
User-agent: *
Disallow: /cgi-bin/
Disallow: /images/
Disallow: /themes/
Disallow: /admin/
Disallow: /staff/
Disallow: /static/
Disallow: /downloads/
Yes.
How to prevent robots (Google Bot, MSNBot, Yahoo Slurp! etc.) from displaying cached pages of your site
You can insert the following meta tag using meta tag plugin:
<META NAME=”ROBOTS” CONTENT=”NOARCHIVE NOINDEX”>
podz vs google – brought to you by Warner Bros 🙂
LOL !
Xerocool – freudian slip there maybe ?
Cheers though – I shall use the info 🙂
