Googlebot and RSS

  • gaszer

    (@gaszer)


    19 years, 9 months ago

    I’ll just throw this out there… In looking through the server logs, I notice that the googlebot is constantly looking for index.rdf, rss.xml and atom.xml, so it appears Google is trying to index feeds.
    I know the WP feed is accessed from wp-rss.php, etc. I also notice that Yahoo Feed Seeker chews up the php files no problem. Is there a way to feed googlebot the files it’s looking for? Or should I even be concerned?
    This is the first week of my first install of WP, and I must say I’m impressed. As far as the spiders go, I had just installed WP and within a half an hour, the place was crawling with them. Geez, I didn’t even have any content on there yet…

Viewing 7 replies - 1 through 7 (of 7 total)
  • Mark (podz)

    (@podz)

    19 years, 9 months ago

    Google is trying to grab everything, even if it’s not there. I wouldn’t worry 🙂

    carthik

    (@carthik)

    19 years, 9 months ago

    I think it will still look for those files, no matter what you do, and besides, like podz said, it grabs wp-rss2.php and the others anyway if they are linked-to from your index page.
    You could set up rewrite rules to redirect requests to the right files, and if you use the 301 redirect, google should update the files it looks for since 301 is a “permanently moved” response.

    joshuaclinard

    (@joshuaclinard)

    19 years, 9 months ago

    I had the same thing in my logs a few weeks ago. What I did was redirect all the files that google was looking for to wp-rss2.php

    Thread Starter gaszer

    (@gaszer)

    19 years, 9 months ago

    Thanks for the quick responses. Pardon my ignorance, but how would I go about setting up the redirect requests?

    joshuaclinard

    (@joshuaclinard)

    19 years, 9 months ago

    Put this in your .htaccess file.
    Redirect permanent /blog/index.rdf http://www.yourdomain.com/blog/wp-rss2.php
    One entry like that for each of the files you want to redirect.

    carthik

    (@carthik)

    19 years, 9 months ago

    gaszer, wp-rss2.php is the rss2 file generated by wordpress. for atom, there is wp-atom.php and wp-rss.php is the RSS 1.0 file. wp-rdf.php is the rdf file.
    So you will have to do a bunch of redirect rules, each of which resembles the one described by Joshua, one for each file that google is looking for.

    Thread Starter gaszer

    (@gaszer)

    19 years, 9 months ago

    Gracias!

Viewing 7 replies - 1 through 7 (of 7 total)
  • The topic ‘Googlebot and RSS’ is closed to new replies.