WordPress.org

Ready to get started?Download WordPress

Forums

Google Webmaster Crawl Error : Robot.txt (2 posts)

  1. dj007
    Member
    Posted 12 months ago #

    Hello,

    Yesterday I changed my old domain url to a new domain url that I am hosting with WordPress through Google. I did the URL redirect along with the address change on Google Webmaster and also moved my existing folder to the new domain's root. But today I received a message on Google Webmaster stating that they couldn't crawl through my robot.txt file. This happened to me before and the next day they crawled it perfectly fine. I am just anxious and curious if I am even doing this address change correctly.

    Here is my what my robot.txt is showing me, Is this correct? or is there something internally that I have to fix? any help would be appreciated! Thanks

    http://www.newdomainurl.com/robots.txt

    # Added by SEO Ultimate's Link Mask Generator module
    User-agent: *
    Disallow: /go/
    # End Link Mask Generator output
    
    User-agent: *
    Disallow: /wp-admin/
    Disallow: /wp-includes/
  2. Krishna
    Volunteer Moderator
    Posted 12 months ago #

    Try using only:

    User-agent: *
    Disallow: /wp-admin/
    Disallow: /wp-includes/

Reply

You must log in to post.

About this Topic