Google XML Sitemaps
Google not crawling (1 post)

  1. Sdarter
    Posted 3 years ago #

    We got an error on a new site:

    User-agent: *
    Disallow: /wp-admin/
    Disallow: /wp-includes/

    and Google was saying it couldn't access the site. Privacy settings were turned to allow indexing.

    When we unchecked the box "Add sitemap URL to the virtual robots.txt file" it started working. Can you please explain? Should we always uncheck that box? Why is it checked by default if it blocks the indexing?

    Thanks for your help in understanding this!


Topic Closed

This topic has been closed to new replies.

About this Plugin

  • Google XML Sitemaps
  • Frequently Asked Questions
  • Support Threads
  • Reviews

About this Topic