Get a Google account and use the Google Webmaster Tools to run a diagnostic on your site. It’ll show if there are any crawl errors.
Also, log into WP and under Settings -> Privacy, make sure your WP isn’t blocking search engines.
I have the same problem. New posts are not indexed. Have a google account, no crawl errors showing. Google ran throught the site on Nov 1st but the new posts are not indexed. WP settings are correct.
Check your page source and be sure you don’t still have a “no index, no follow” meta tag. And check for a robots.txt file that is blocking bots. Other than that, wait on Google.
Already did. Don’t have a nofollow,noindex tag. Wait for what? We were indexed immediately before, popped up number 1 after all new posts, now only the old ones show up.
Try turning up Google’s crawl rate in your sitemap account. Might work. Google also likes good original content, too, not aggregator content.
We only have original content. I can try modifying the crawl rate but they crawled the site yesterday.
I’d like to know what happened. I have read other similar posts as well, something happened…
nazsit: Google can be very slow to index new sites, sometimes taking a week or more to do so. They did recently announce that they are now using site feeds for discovering and indexing, so make sure that you have WordPress set to ping update services (Dashboard -> Settings -> Writing). I’m not sure if they gather info from Google Reader, but you could try subscribing to your site in Google Reader as well.
http://googlewebmastercentral.blogspot.com/2009/10/using-rssatom-feeds-to-discover-new.html
http://codex.wordpress.org/Update_Services
To see if your site is in Google, try this search with your own domain: site:example.com
iridiax, Really good additonal info re: Goggle and feeds…..
The blog is NOT new. It’s been around – and well indexed – since 2006.
Google just recently stopped indexing new posts. Google is crawling the site just not adding the new posts.
If your site is not new and Google stops indexing or starts dropping indexed pages from its index, then check with Google Webmaster Tools for any site issues (malware, penalties, URL unreachable errors, robots.txt problems, etc.). I had this exact problem at one web host due to network unreachable and robots.txt file unreachable errors.
have the same problem http://roshanratnayake.wordpress.com/
please advise how i can fix
Would love to get some help on this same topic.
My blog is not being indexed by search engines and as it turns out google is saying that they robot.txt file is unreachable.
This is the error message that I get for this URL:
http://thetravelmonster.com/robots.txt
Internal Server Error
The server encountered an internal error or misconfiguration and was unable to complete your request.
Please contact the server administrator, [no address given] and inform them of the time the error occurred, and anything you might have done that may have caused the error.
More information about this error may be available in the server error log.
Additionally, a 500 Internal Server Error error was encountered while trying to use an ErrorDocument to handle the request.
Can someone help?
Also, I host my blog through network solutions just in case that is a factor.
Thanks so much!
