WordPress.org

1. BPA
   Member
   Posted 1 year ago #

   Hi,

   I've been using this URL structure (permalink) for my site's pages:
   mysite.com/news/something-something
   I'm not sure if the default URLs have been 301 redirected and if not how can I do this.

   I've added this to my .htaccess:

   RewriteCond %{THE_REQUEST} ^GET\ /.*\;.*\ HTTP/
   RewriteCond %{QUERY_STRING} !^page_id=$
   RewriteRule .* http://mobiledealstop.co.uk%{REQUEST_URI}? [R=301,L]

   but Google still shows the ?page_id URLs under Webmaster tools/Restricted by robots.txt, meaning that these are still accessible despite disabling them in the robots.txt file.

   I will appreciate any advice

2. SpaceDogDeveloper
   Member
   Posted 1 year ago #

   To check a 301 redirect, you need to look at the header conversation of a browser. If you use Firefox, there's a great plugin called Live HTTP Headers.

   If you are redirecting a page to retain the PageRank, DON'T disallow it via robots.txt, as that will simply remove the pages from being spidered (which, I'm assuming is the reason for the redirect).

   Also, if the ?page_id URLs are showing up in Webmaster tools under the "Restricted by Robots.txt" section, then your robots.txt is actually working. Keep in mind, robots.txt doesn't mean that Google won't spider those pages. It just means they won't index them. The disallow: parameter is not a magic bullet to keep out spiders, its just a recommendation not to index them. If you really want to keep Google and other spiders away from a page, password protect the folder. That's really the only way to do it.

3. BPA
   Member
   Posted 1 year ago #

   Yes, that's exactly what I suspect - the strangest thing is that these ?page_id URLs appeared in my WT a week ago and have no idea why now?! The site has been online for 3 years and haven't changed a thing. I've been using the same URL structure since the beginning... I'll check the Firefox plugin, thanks for tip ;)

   P.S. actually I've got this plugin - I just can not see how to check if a ?page_id URL has been redirected - the WP rewrites these and opens the correct one in the browser. Hopefully the above code is correct and Google will stop spidering and will drop these URLs

4. SpaceDogDeveloper
   Member
   Posted 1 year ago #

   If your site has had those URLs for three years, don't expect Google to update that anytime soon. Google is all about age and trust, and they'll keep an old version of a URL in the index for quite awhile. I've seen Google hang on to old versions for literally half a year in some cases. As long as the pages redirect (if you click on the old version in the index and it goes to the new one), then you're just fine. I strongly suggest you DON'T disallow these old urls as you can accidentally remove the new ones from the index.

   When you're using a 301 you are essentially telling Google "This page has been permanently moved to here", much like forwarding your mail from your old apartment to your new apartment. And to use the post office example, disallowing would be like telling the post office "stop all mail to the address of my old apartment", which would send that mail to the dead letter office.

   Hopefully that made sense :)

5. SpaceDogDeveloper
   Member
   Posted 1 year ago #

   Wait a sec. I just read that you've been using the (mysite.com/news/something-something) permalink structure for three years? If that is true, it is very strange.. don't suppose you could send me your URL?

6. BPA
   Member
   Posted 1 year ago #

   It's the wp_dropdown_pages( $args ); which generates these URLs. I started using it last month and it never crossed my mind that this would do such a damage! I've removed it from the site and hopefully this will solve the problem :)

Topic Closed

This topic has been closed to new replies.