WordPress.org

1. c3lt1957
   Member
   Posted 2 years ago #

   Hi Guys,

   I have just noticed that Google is picking up a virtual robot.txt file from my site that says;

   User-agent: *
   Disallow: /

   I deleted my original robot.txt file on the instruction of the Google XML site map plugin, which states that the virtual robot.txt produced by WordPress will be used !

   My question is this, surely the entered text in this virtual robot.txt file is incorrect..? would it not prevent my site from being crawled by any search engine bot..?

   Can somebody clarify this point please..?

   Steve

2. whooami
   Member
   Posted 2 years ago #

   settings --> privacy -- you set that up during the initial install.

3. KimTaylor
   Member
   Posted 1 year ago #

   Hi I have the same problem google webmaster tools show that the robots.txt (the virtual one)

   says:

   User-agent: *
   Disallow:

   Sitemap: http://yeast-infection.just-4-women.org/sitemap.xml.gz

   Sadley i dont know what this means but am worried that bots are being disallowed from crawling the site.

   the privicy settings are set to allow bots to view.

   so what do i do - if anything?

Topic Closed

This topic has been closed to new replies.