Crawl errors on my site

  • nixgadgets

    (@nixgadgets)


    11 years ago

    Hi guys,

    I have Yoast SEO installed on my blog to ease the process of setting up seo. And it appears to be generating few crawl errors specially with the “archive” links. I’m not entirely sure whats causing these errors as I’m using permalinks.

    I use the recommended permalink structure “/%postname%”

    Examples of the crawl errors as shown below,

    archives/author/kasunt/page/4 404 4/12/13

Viewing 6 replies - 1 through 6 (of 6 total)
  • Krishna

    (@1nexus)

    11 years ago

    Where did you find the errors? At Webmaster Tools? Duplicates or something else?

    Thread Starter nixgadgets

    (@nixgadgets)

    11 years ago

    Yes it is webmaster tools. Not duplicates as far as I can tell. Linked from for this particular crawl error shows the following URLs which obviously doesn’t exist.

    [links moderated]

    Thread Starter nixgadgets

    (@nixgadgets)

    11 years ago

    And this is my robots.txt

    User-Agent: *
    Disallow: /feed
    Disallow: /*/feed
    Disallow: /wp-content/plugins/

    Krishna

    (@1nexus)

    11 years ago

    They are duplicates. They appear in their own URLs, no? You can safely tell Google to ignore them.

    Thread Starter nixgadgets

    (@nixgadgets)

    11 years ago

    So you mean to say that I should just add “Disallow: /archives” ?

    Thanks in advance.

    Krishna

    (@1nexus)

    11 years ago

    Please thoroughly read and follow the instructions here:
    http://support.google.com/webmasters/bin/answer.py?hl=en&answer=66359

    If you are using any SEO plugin or if your theme has SEO settings, configure them correctly so that you won’t run into such problems.

Viewing 6 replies - 1 through 6 (of 6 total)
  • The topic ‘Crawl errors on my site’ is closed to new replies.