Hide development site from search engines (4 posts)

  1. TheJoyfulOrganizer
    Posted 5 years ago #


    I have been developing a new company website with wordpress, and I see that it is already coming up in search engines. A good thing for down the road, a bad thing until it is done. Is there any way to stop this form happening? THanks!

  2. mtw28
    Posted 5 years ago #

    If you want to exclude all the search engine spiders from your entire domain, you would add a robots.txt file your website folder and put the following text in it:

    User-agent: *
    Disallow: /

    Use with care as the site won't get indexed by search engines while the file is still there. See here for more information on robots.txt.

  3. TheJoyfulOrganizer
    Posted 5 years ago #

    Here's the thing, if I have installed WordPress on my hosting server that contains my current company site, will that not stop all search engine spiders and impact my current working site too? So that would probably be a bad thing overall, right?

  4. mtw28
    Posted 5 years ago #

    It should only apply to subfolders, so if wordpress is installed to http://www.yourdomain.com/wordpress/ and you put robots.txt in http://www.yourdomain.com/wordpress/robots.txt, it should be fine.

    To identify each file you want to exclude, and not folders and domains, you can also put this information in meta fields in the html or wordpress theme template directly, for example:


    More info on the meta no robots approach here.

    Another idea would be to forward all traffic coming to your under-development site to the new site, using something like this:

    CONTENT="5; URL=autoforward_target.html">

Topic Closed

This topic has been closed to new replies.

About this Topic


No tags yet.