Question about robots.txt (4 posts)

  1. hesed2
    Posted 11 years ago #

    I have one Page (as in a WP Page) on my site that I don't want crawled. If the page is http://www.mysite.com/test/ and I set my robots.txt file so the the /test/ directory isn't crawled, will this work since it's a virtual directory?

  2. Beel
    Posted 11 years ago #

    The crawling is done by links so it should be fine if I understand your question correctly. You can also use is_page() in the header to set a "no index" meta tag for particular pages.

  3. James Huff
    Support Representative
    Posted 11 years ago #

    Yes, it will work.

    ::Darn, too slow again.::

  4. hesed2
    Posted 11 years ago #

    Cool, thanks guys.

Topic Closed

This topic has been closed to new replies.

About this Topic


No tags yet.