WordPress.org

Ready to get started?Download WordPress

Forums

[resolved] Robots.txt removal problems - can't figure this one out (4 posts)

  1. havanajim
    Member
    Posted 2 years ago #

    Hi..am a little confused since I ticked the box to allow search engines to now crawl my site (previously asked for them not to) but Google webmaster tools is telling me I still have robots.txt blocking them so am unable to submit the sitemap.

    Any ideas please?

  2. havanajim
    Member
    Posted 2 years ago #

    UPDATE:

    Google has now accepted 5 pages out of the 7 pages in the site map but I don't see my homepage indexed with it's meta tag so something seems to be wrong? (My site is http://containerforsale.co.uk)

    I followed the suggestion to write the robots.txt the following way:

    User-agent: *

    Disallow:

    Sitemap: http://containerforsale.co.uk/sitemap.xml

    But have since had a reply post on Google webmaster forum suggesting that the above robots.txt file is invalid and may cause Google not to crawl the site. Is this correct? If so how should it be written please?

    Thanks for any help.

  3. adpawl
    Member
    Posted 2 years ago #

    Sitemap: http://containerforsale.co.uk/sitemap.xml
    
    User-agent: *
    Allow: /
  4. havanajim
    Member
    Posted 2 years ago #

    Adpawl, thanks for your reply..

Topic Closed

This topic has been closed to new replies.

About this Topic