Yeah ok I know I can use robots. The problem is the only way to block an individual page is to insert a <META> tag into the HTMl code into the page.
The problem is I don’t know how to access the HTML code of the page to put it into the <HEAD> section. If someone can help me figure out how to edit the HTML of a wordpress page that would be great.
Yeah ok I know I can use robots. The problem is the only way to block an individual page is to insert a <META> tag into the HTMl code into the page.
thats not true, and thats why I directed you to do some more reading.
using a robots.txt file prevents a url from being spidered, negating the need for any further fiddling.
if you cant take my word for it — take Google’s :
On getting a url removed:
… use a robots.txt file or meta noindex tag to block crawlers from accessing your content.
–
all emphasis is mine.
Ok yes it is clear that you are telling my to RTFM. I have been doing that all night and am not understanding how to accomplish what I’m trying to do. Sorry if I’m sounding frustrated.
I’ve read this (http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=40360) and it says to block a certain page I need to do this
# To block a page, list the page.
Disallow: /private_file.html
Except I look on this page (http://codex.wordpress.org/Pages)
and it says
What Pages are Not:
* Pages are not files. They are stored in your database just like Posts are.
The thing I’m not understanding is that google says if I want to block a page I have to block a file. But then wordpress does not store it’s pages as files so there is an obvious problem there and I don’t know where to go from there.
Sorry if I’m a total dummy but if I was just blocking a html file on the FTP I would be fine. My problem comes up because it says the page is stored in a database and isn’t an actual file. So I don’t know what URL to put to block that.
P.S. Sorry if I sound frustrated, but I am and maybe it’s because I don’t understand how wordpress stores it’s files or whatever but I have been working on this for several hours just trying to get this one thing taken care of.
it doesnt matter how wordpress stores files — theyre served, just like any other urls.
http://www.domain.com/mypage
is the same url .. whether or not wordpress creates it or not.
you have a page, its name is thing. the permalink to it is http://www.mywordpress.com/thing
what do you put in your robots.txt?
Disallow: /thing/
very simple.