WordPress.org

Ready to get started?Download WordPress

Forums

Lot of 404 Errors (17 posts)

  1. vinoth06
    Member
    Posted 9 months ago #

    Hello,

    My site getting around 5000+ '404' errors. Please check the attached image.

    http://i.imgur.com/dtMGM6h.jpg

    1. Am having tag, but I don't have such url.

    Google webmaster link
    "tag/768x1024/download/fd.php?file=paul-2/paul-2-960x540.jpg"

    But Actual one will be
    "MY_DOMAIN_NAME/download/fd.php?file=paul-2/paul-2-960x540.jpg"

    Authors,

    Google webmaster link
    "author/kelly/download/fd.php?file=robot-spider-01/robot-spider-1680x1260.jpg"
    But Actual one will be
    "MY_DOMAIN_NAME/download/fd.php?file=robot-spider-01/robot-spider-1680x1260.jpg"

    Category

    Google webmaster link
    "category/flowers/download/fd.php?file=butterfly-nature/nature-butterfly-1152x864.jpg"

    But Actual one will be
    "MY_DOMAIN_NAME/download/fd.php?file=butterfly-nature/nature-butterfly-1152x864.jpg"

    How to solve this 404 Errors. Please suggest me

  2. vinoth06
    Member
    Posted 9 months ago #

    I found this method on internet. Will it suitable for this problem ?

    add the following code in the <head> section to the header.php page for your theme.
    
    <?php if
    (
    is_search() ||
    is_archive() ||
    is_404() ||
    is_feed() ||
    is_paged()
    ){ ?>
    <meta name="robots" content="noindex,follow" />
    <?php } ?>
  3. Krishna
    Volunteer Moderator
    Posted 9 months ago #

    Do you have the above quoted URLs in your sitemap submitted to Google (Webmaster Tools)?
    Can you post your site URL?

  4. vinoth06
    Member
    Posted 9 months ago #

    Some of the links

    http://5dwallpapers.com/category/sports/download/fd.php?file=zidane-player/zidane-soccer-player-240x400.jpg
    
    http://5dwallpapers.com/tag/800x480/page/52/
    
    http://5dwallpapers.com/tag/2048x2048/page/3/
  5. Krishna
    Volunteer Moderator
    Posted 9 months ago #

    Do you have the above quoted URLs in your sitemap submitted to Google (Webmaster Tools)?

    Here is what you have in your robots.txt file:

    User-Agent: *
    Disallow:
    Disallow: /wp-includes/
    Disallow: /wp-admin/
    Disallow: /wp-content/
    Disallow: /cgi/
    
    User-Agent: Googlebot-Image
    Allow: /download/
    Allow: /wp-content/uploads/

    Instead of the above, try something like this:

    User-Agent: *
    Disallow: /wp-includes/
    Disallow: /wp-admin/
    Disallow: /cgi/
    Disallow: /tag/
  6. vinoth06
    Member
    Posted 9 months ago #

    If we Disallow the tag, will it good for my site to crawl the tags ? Please suggest.

  7. If you disallow tags, Google won't crawl /tag/ in URLs.

    download/fd.php?

    That concerns me more. Do you have something that allows downloads?

  8. vinoth06
    Member
    Posted 9 months ago #

    Yes, download/fd.php will make the image to be downloadable. So blocking download/fd.php file won't be problem. But am having all files in download so blocking complete download/ folder will block all images.

  9. Krishna
    Volunteer Moderator
    Posted 9 months ago #

    so blocking complete download/ folder will block all images.

    Look at what you posted first. If that is correct, you need not block download links, but only those with tag, category, author, etc. Also see what you have in your sitemap.

  10. vinoth06
    Member
    Posted 9 months ago #

    @krishna: At sitemap I haven't added tags, category and authors.

    Actually the problem is, I don't have this exact url

    http://5dwallpapers.com/tag/768x1024/download/fd.php?file=paul-2/paul-2-960x540.jpg

    This download/fd.php should work only on individual post not either in tags, category and author. But Google crawled like

    http://5dwallpapers.com/tag/768x1024/download/fd.php?file=paul-2/paul-2-960x540.jpg

  11. Krishna
    Volunteer Moderator
    Posted 9 months ago #

    The robots.txt suggested by me above will take care of the issue. Similarly, you can also disallow category, author, etc. that are not in your sitemap. Actually, the posts that appear with URLs of tags, category, etc. also appear with their post URLs and hence you don't need them indexed separately. Google bots crawl them just because they have separate URLs and you have not prevented bots from crawling using your robots.txt file.

  12. vinoth06
    Member
    Posted 9 months ago #

    @krishna: my site has been indexed with many tags. I can see the url in Google. So if I disallow that tags, will Google keep that link or delete that link ?

    for example,

    I have seen in search engine that

    http://5dwallpapers/tags/480x320
    has been indexed.

    If I disallow tags, what would happen ? Will Google delete the above link or will be retained even after disallow tags in robot.txt ?

  13. Krishna
    Volunteer Moderator
    Posted 9 months ago #

    You may change your robots.txt as desired, submit a new sitemap and allow time (can be soon or take a few days) for re-indexing. Then follow up for further support at your Webmaster Tools account. In all probability, changes will be as desired by you.

  14. vinoth06
    Member
    Posted 8 months ago #

    Update,

    I found the same kind of link

    http://5dwallpapers.com/tag/2800x2100/download/fd.php?file=animal/elephant-640x960.jpg

    on

    http://5dwallpapers.com/tag/2800x2100/feed/

    Check this image from webmaster,

    http://i.imgur.com/L5fQXli.jpg

    Hope this will give some solution for my problem

  15. Krishna
    Volunteer Moderator
    Posted 8 months ago #

    So, did you disallow tag in your robots.txt file?

  16. vinoth06
    Member
    Posted 8 months ago #

    Four days before, I have added this code,

    add the following code in the <head> section to the header.php page for your theme.

    <?php if
    (
    is_search() ||
    is_archive() ||
    is_404() ||
    is_feed() ||
    is_paged()
    ){ ?>
    <meta name="robots" content="noindex,follow" />
    <?php } ?>

    Suddenly my search has been dropped from 300 visitors per day to 120 visitors. After removing this code now am getting back that 300 visitors per day.

    So blocking tag may sudden drop in search engine result. Kindly suggest me.

  17. Krishna
    Volunteer Moderator
    Posted 8 months ago #

    I don't think blocking tag will drop your traffic. The drop that you report above is mainly caused by blocking (noindex) archive and pages, as far as I understand. Again as a test, you may try various options as you can never know (or get) the algorithms Google use.

Reply

You must log in to post.

About this Topic