WordPress.org

Support

Support » Miscellaneous » Lot of 404 Errors

Lot of 404 Errors

  • Hello,

    My site getting around 5000+ ‘404’ errors. Please check the attached image.

    View post on imgur.com

    1. Am having tag, but I don’t have such url.

    Google webmaster link
    “tag/768×1024/download/fd.php?file=paul-2/paul-2-960×540.jpg”

    But Actual one will be
    “MY_DOMAIN_NAME/download/fd.php?file=paul-2/paul-2-960×540.jpg”

    Authors,

    Google webmaster link
    “author/kelly/download/fd.php?file=robot-spider-01/robot-spider-1680×1260.jpg”
    But Actual one will be
    “MY_DOMAIN_NAME/download/fd.php?file=robot-spider-01/robot-spider-1680×1260.jpg”

    Category

    Google webmaster link
    “category/flowers/download/fd.php?file=butterfly-nature/nature-butterfly-1152×864.jpg”

    But Actual one will be
    “MY_DOMAIN_NAME/download/fd.php?file=butterfly-nature/nature-butterfly-1152×864.jpg”

    How to solve this 404 Errors. Please suggest me

Viewing 15 replies - 1 through 15 (of 16 total)
  • I found this method on internet. Will it suitable for this problem ?

    add the following code in the <head> section to the header.php page for your theme.
    
    <?php if
    (
    is_search() ||
    is_archive() ||
    is_404() ||
    is_feed() ||
    is_paged()
    ){ ?>
    <meta name="robots" content="noindex,follow" />
    <?php } ?>

    Do you have the above quoted URLs in your sitemap submitted to Google (Webmaster Tools)?
    Can you post your site URL?

    Some of the links

    http://5dwallpapers.com/category/sports/download/fd.php?file=zidane-player/zidane-soccer-player-240x400.jpg
    
    http://5dwallpapers.com/tag/800x480/page/52/
    
    http://5dwallpapers.com/tag/2048x2048/page/3/

    Do you have the above quoted URLs in your sitemap submitted to Google (Webmaster Tools)?

    Here is what you have in your robots.txt file:

    User-Agent: *
    Disallow:
    Disallow: /wp-includes/
    Disallow: /wp-admin/
    Disallow: /wp-content/
    Disallow: /cgi/
    
    User-Agent: Googlebot-Image
    Allow: /download/
    Allow: /wp-content/uploads/

    Instead of the above, try something like this:

    User-Agent: *
    Disallow: /wp-includes/
    Disallow: /wp-admin/
    Disallow: /cgi/
    Disallow: /tag/

    If we Disallow the tag, will it good for my site to crawl the tags ? Please suggest.

    If you disallow tags, Google won’t crawl /tag/ in URLs.

    download/fd.php?

    That concerns me more. Do you have something that allows downloads?

    Yes, download/fd.php will make the image to be downloadable. So blocking download/fd.php file won’t be problem. But am having all files in download so blocking complete download/ folder will block all images.

    so blocking complete download/ folder will block all images.

    Look at what you posted first. If that is correct, you need not block download links, but only those with tag, category, author, etc. Also see what you have in your sitemap.

    @krishna: At sitemap I haven’t added tags, category and authors.

    Actually the problem is, I don’t have this exact url

    http://5dwallpapers.com/tag/768x1024/download/fd.php?file=paul-2/paul-2-960x540.jpg

    This download/fd.php should work only on individual post not either in tags, category and author. But Google crawled like

    http://5dwallpapers.com/tag/768x1024/download/fd.php?file=paul-2/paul-2-960x540.jpg

    The robots.txt suggested by me above will take care of the issue. Similarly, you can also disallow category, author, etc. that are not in your sitemap. Actually, the posts that appear with URLs of tags, category, etc. also appear with their post URLs and hence you don’t need them indexed separately. Google bots crawl them just because they have separate URLs and you have not prevented bots from crawling using your robots.txt file.

    @krishna: my site has been indexed with many tags. I can see the url in Google. So if I disallow that tags, will Google keep that link or delete that link ?

    for example,

    I have seen in search engine that

    http://5dwallpapers/tags/480x320
    has been indexed.

    If I disallow tags, what would happen ? Will Google delete the above link or will be retained even after disallow tags in robot.txt ?

    You may change your robots.txt as desired, submit a new sitemap and allow time (can be soon or take a few days) for re-indexing. Then follow up for further support at your Webmaster Tools account. In all probability, changes will be as desired by you.

    Update,

    I found the same kind of link

    http://5dwallpapers.com/tag/2800x2100/download/fd.php?file=animal/elephant-640x960.jpg

    on

    http://5dwallpapers.com/tag/2800x2100/feed/

    Check this image from webmaster,

    http://i.imgur.com/L5fQXli.jpg

    Hope this will give some solution for my problem

    So, did you disallow tag in your robots.txt file?

    Four days before, I have added this code,

    add the following code in the <head> section to the header.php page for your theme.

    <?php if
    (
    is_search() ||
    is_archive() ||
    is_404() ||
    is_feed() ||
    is_paged()
    ){ ?>
    <meta name=”robots” content=”noindex,follow” />
    <?php } ?>

    Suddenly my search has been dropped from 300 visitors per day to 120 visitors. After removing this code now am getting back that 300 visitors per day.

    So blocking tag may sudden drop in search engine result. Kindly suggest me.

Viewing 15 replies - 1 through 15 (of 16 total)
  • The topic ‘Lot of 404 Errors’ is closed to new replies.