WordPress.org

Forums

Google Webmaster Crawl Error : Robot.txt (2 posts)

  1. dj007
    Member
    Posted 1 year ago #

    Hello,

    Yesterday I changed my old domain url to a new domain url that I am hosting with WordPress through Google. I did the URL redirect along with the address change on Google Webmaster and also moved my existing folder to the new domain's root. But today I received a message on Google Webmaster stating that they couldn't crawl through my robot.txt file. This happened to me before and the next day they crawled it perfectly fine. I am just anxious and curious if I am even doing this address change correctly.

    Here is my what my robot.txt is showing me, Is this correct? or is there something internally that I have to fix? any help would be appreciated! Thanks

    http://www.newdomainurl.com/robots.txt

    # Added by SEO Ultimate's Link Mask Generator module
    User-agent: *
    Disallow: /go/
    # End Link Mask Generator output
    
    User-agent: *
    Disallow: /wp-admin/
    Disallow: /wp-includes/
  2. Krishna
    Volunteer Moderator
    Posted 1 year ago #

    Try using only:

    User-agent: *
    Disallow: /wp-admin/
    Disallow: /wp-includes/

Topic Closed

This topic has been closed to new replies.

About this Topic