WordPress.org

Ready to get started?Download WordPress

Forums

[resolved] crawl error for posts which no longer exist (7 posts)

  1. harish_das
    Member
    Posted 3 years ago #

    Hi
    I have been getting crawl errors for posts which no longer exist on my blog. It comes as not avaialable. And it come from many links. So error list keeps on increasing. I use Google XML sitemaps to generate sitemaps. I deleted my old sitemap and created a new one manually but still no success. Do I need to delete any other files whenever I delete or modify any post?
    Thanks

  2. Peter vanDoorn
    Member
    Posted 3 years ago #

    Once Google knows about a page it will periodically check to see that it is still there (that's what the crawling is all about).

    It will get the message eventually that it doesn't exist, but it does usually takes a couple of weeks!

    Patience is not just a card game.

    Peter

  3. pete_398
    Member
    Posted 3 years ago #

    It will take a long time, more than likely many months. We still have old links to some files appear in Google after a long time, and the google bot visits a few times a day.

    We wrote a small php script that when an (old) post id was attempted to be accessed, it did a 301 (permanent redirect) to the new link. That way, it caters for google and all visitors, as you may have links from other sites.

  4. jpomerantz
    Member
    Posted 2 years ago #

    Pete - can you share the php? I am having a similar issue with non-existing pages that are months old.

    Thanks

    Jay
    http://www.theparentschool.net

  5. pete_398
    Member
    Posted 2 years ago #

    Jay, the following is not the entire script, but a small section of code. You would normally place this in your '404' (not found) file.

    <?php
    
    $http_host = $_SERVER["HTTP_HOST"];
    $request_uri = $_SERVER['REQUEST_URI'];
    
    //For posts/pages which no longer exist, setup array to point to new url
    $old_url = 'http://' . $http_host . strtolower($request_uri);
    
    $urls = array(
       'http://example.com/wordpress/?p=1' => 'http://example.com/?p=1',
       'http://example.com/wordpress/?p=13' => 'http://example.com/?p=3',
       'http://example.com/wordpress/?p=18' => 'http://example.com/?p=4',
       'http://example.com/wordpress/?p=22' => 'http://example.com/?p=5',
       'http://example.com/wordpress/?p=29' => 'http://example.com/?p=6',
       'http://example.com/wordpress/?p=34' => 'http://example.com/?p=7',
       'http://example.com/wordpress/?p=39' => 'http://example.com/?p=8',
       'http://example.com/wordpress/?p=43' => 'http://example.com/?p=9'
       );
    
    if (array_key_exists($old_url, $urls))
    {
    	header( "HTTP/1.1 301 Moved Permanently" );
    	header("Location: $urls[$old_url]"); /* Redirect browser */
    	/* Make sure that code below does not get executed when we redirect. */
    	exit();
    }
    
    //Other code below ...
    
    ?>
  6. jpomerantz
    Member
    Posted 2 years ago #

    Thanks Pete - am I correct to assume that what you provided will be what I need - I just need to change the page names and redirects?

  7. pete_398
    Member
    Posted 2 years ago #

    Each site will be different. I had a load of other code that was pretty much relevant to only that site, so didn't post it.

    I'm pretty sure that that code will work okay, but I haven't tested it. Best to place it in a php file, modify it with 'echo' statements to display those variables, then you will be confident it is doing what you want it to do.

    You may need to make sure the echos are before e redirect I think, and also before a 'header' from memory.

    If it doesn't work, post here, and myself or someone else will be able to help hopefully.

Topic Closed

This topic has been closed to new replies.

About this Topic