Major Problem solved with WordPress & Google

oztwo
(@oztwo)

15 years, 7 months ago

After getting WordPress and setting up my site, I soon found problems with Google not indexing my site. When reading up on the problem, I couldn’t really find an answer to the problem other than people saying to just give it time. With my passed sites, I found Google to be able to index the sites with in minutes.

Well, I gave it a week and a day and still no action from Google. Google was crawling my site like mad:

Number of pages crawled per day
Maximum 359
Average 194
Minimum 45

I setup a Google Webmaster tools account, yet Google just would not index my site. This being my first time using WordPress I was starting to wonder if maybe Google had something with WordPress blogs or even the new domain I bought.

Then one day as I was in my car driving home I had an idea of what the problem could be. Racing home to see if my vision was right, I found it was. After fixing the problem, Google index my site within seconds.

The problem?

<?php wp_get_archives(‘type=monthly&format=link’); ?>

What this function does is put in meta links to all archives on a given site such as:

<link rel=”alternate” type=”

This may seem like a great idea, but it’s not at all. What is happening is that you are more or less putting the crawler in a big loop. It’s finding these links in your header and going to them and not even getting to your site which in turn means your site isn’t going to get indexed anytime soon.

So if you find that you are having problems, remove the above function or any other function that adds this type of data in your header. These functions can be found in the theme’s header.php file. An example of a header.php file before cleaning it up looked like this:

<link rel=”alternate” type=”application/rss+xml” title=”RSS 2.0″ href=”<?php bloginfo(‘rss2_url’); ?>” />
<link rel=”alternate” type=”application/rss+xml” title=”RSS 2.0″ href=”<?php bloginfo(‘rss2_url’); ?>” />
<link rel=”alternate” type=”text/xml” title=”RSS .92″ href=”<?php bloginfo(‘rss_url’); ?>” />
<link rel=”alternate” type=”application/atom+xml” title=”Atom 0.3″ href=”<?php bloginfo(‘atom_url’); ?>” />
<link rel=”pingback” href=”<?php bloginfo(‘pingback_url’); ?>” />
<?php wp_get_archives(‘type=monthly&format=link’); ?>

I would removed all four such lines of code. Again, after I did remove it, seconds later, Google indexed my site. When I say seconds, I mean seconds. After uploading the header.php file and then going back to Google Webmaster Tools Google showed that it was indexed.

I hope this helps all that may be having problems trying to get their site indexed.

Viewing 11 replies - 1 through 11 (of 11 total)

wendel-brume
(@wendel-brume)

15 years, 7 months ago
Interesting. Here’s what’s in my header.php:
```
<link href="<?php bloginfo('stylesheet_url'); ?>" rel="stylesheet" type="text/css" media="screen" />
<link rel="alternate" type="application/rss+xml" title="<?php bloginfo('name'); ?> RSS Feed" href="<?php bloginfo('rss2_url'); ?>" />
<link rel="shortcut icon" type="image/x-png" href="<?php bloginfo('template_url'); ?>/favicon.png" />
<link rel="pingback" href="<?php bloginfo('pingback_url'); ?>" />
<?php wp_head(); ?>
```
The stylesheet is needed, the pointer to the favicon can’t hurt, but I don’t know how big an effect removing the other two would have. What’s the downside of removing them?
iridiax
(@iridiax)

15 years, 7 months ago

OZtwo: I don’t see anything wrong in the header code you posted, it is the exact same as what is in the fully functional and tested WordPress classic theme. However, I think it’s okay to get rid of <?php wp_get_archives('type=monthly&format=link'); ?> if you want. Are you using any sort of custom archiving plugin that could have conflicted?

If you are having trouble with Google’s crawling or indexing, sign up with Google Webmaster Tools and check later to see if they report any errors.

Thread Starter oztwo
(@oztwo)

15 years, 7 months ago

All I know is that after running into this issue and then doing a simple Google search, I found this to be a very common problem with WordPress users. Google simply would not index their sites. What I was finding very odd was that the google bots were hitting my site like crazy but nothing was happening. Once I removed the line:

<?php wp_get_archives(‘type=monthly&format=link’); ?>

Google instantly added my site. One interesting note as well, I have a test site still up and Google still has not indexed that site. So I have two sites. One that has the above line: No Index (3 months old), and the second main site which I removed the above line and Google is now indexing.

At the end it makes a lot of since. As I put in my last post, I just could get what was going on. Then I thought about how WordPress was working and thought maybe, just maybe it had a lot of unneeded code in the <head> area. Sure enough when I looked at my code, there it was: all my links to all my archives in the header area. So as I was guessing, google was in a simple loop. Checking the links in the header and NEVER making it to the main page because of the links in the header which seemed to be putting google into a loop. Again, look at the stats in my last message. google was hitting my site hard. Now my site is up and going. Being index and the sitemap.xml entries show that they all have been checked. (showing before that it found the file, but was still checking them).

So there it is.

Thread Starter oztwo
(@oztwo)

15 years, 7 months ago

<link href=”<?php bloginfo(‘stylesheet_url’); ?>” rel=”stylesheet” type=”text/css” media=”screen” />
<link rel=”alternate” type=”application/rss+xml” title=”<?php bloginfo(‘name’); ?> RSS Feed” href=”<?php bloginfo(‘rss2_url’); ?>” />
<link rel=”shortcut icon” type=”image/x-png” href=”<?php bloginfo(‘template_url’); ?>/favicon.png” />
<link rel=”pingback” href=”<?php bloginfo(‘pingback_url’); ?>” />
<?php wp_head(); ?>

Yes, that code should be ok.

thunksalot
(@thunksalot)

15 years, 6 months ago

I think this must be the problem that I’m having. At least, I can’t find any other way to explain the fact that Google hasn’t indexed my site. But, I don’t appear to have any of the offending lines mentioned above in my header. Is there something else in my header that could be doing it:

<head profile=”http://gmpg.org/xfn/11″>
<meta http-equiv=”Content-Type” content=”<?php bloginfo(‘html_type’); ?>; charset=<?php bloginfo(‘charset’); ?>” />

<title><?php bloginfo(‘name’); ?> <?php if ( is_single() ) { ?> » Blog Archive <?php } ?> <?php wp_title(); ?></title>

<meta name=”generator” content=”WordPress <?php bloginfo(‘version’); ?>” /> <!– leave this for stats –>

<link rel=”stylesheet” href=”<?php bloginfo(‘stylesheet_url’); ?>” type=”text/css” media=”screen” />
<link rel=”alternate” type=”application/rss+xml” title=”<?php bloginfo(‘name’); ?> RSS Feed” href=”<?php bloginfo(‘rss2_url’); ?>” />

<style type=”text/css” media=”screen”>

<?php
// Checks to see whether it needs a sidebar or not
if ( !$withcomments && !is_single() ) {
?>
#page { background: url(“<?php bloginfo(‘stylesheet_directory’); ?>/images/middle900-<?php bloginfo(‘text_direction’); ?>.jpg”) repeat-y top; border: none; background-position: center;}
<?php } else { // No sidebar ?>
#page { background: url(“<?php bloginfo(‘stylesheet_directory’); ?>/images/middle900-ltr.jpg”) repeat-y top; border: none; background-position: center; }
<?php } ?>

</style>

<?php wp_head(); ?>
</head>

ikester
(@ikester)

15 years, 5 months ago

Greetings OZtwo

I tried your solution eliminating by using the comment option in my header.php and remove this line
<?php wp_get_archives(‘type=yearly&format=link’); ?> The only difference with mine was your said monthly and mine yearly.

However, google still is unable to index my posts. Perhaps you have some other suggestions or anyone else please;

site: http://surfinjaco.com

Thank you

jenny867
(@jenny867)

15 years, 1 month ago

OZtwo: Just wanted to say, Thank You so much!! I was having the exact same problem, wracking my brain trying to figure out why Google wasn’t indexing my links. (I even went to the Google Webmaster Forum yesterday and got chewed out by a moderator who claimed I hadn’t searched hard enough for the answer.) No one bothered to ask me what the platform of my site.

I followed your advice, and voila: the very next day, Google had indexed 88 external links. Amazing. I really can’t thank you enough, as this has been a major frustration.

camg
(@camg)

14 years, 9 months ago

Silly question, I found the line of code through the admin section, appearance, editor, then header.php

I deleted the code and hit update, but it keeps re-inserting it.

Naturally I am stuffing it up, so has anyone any ideas on how I should delete the code.

ta

camg
(@camg)

14 years, 9 months ago

Ignore the above post – found the problem

suddenjay
(@suddenjay)

14 years, 8 months ago

I’m having the same problem. But I don’t see teh archives line that’s causing your issues.

Miine are:

<link rel=”shorcut icon” type=”image/x-ico” href=”<?php bloginfo(‘template_url’); ?>/favicon.ico” />
<link href=”<?php bloginfo(‘stylesheet_url’); ?>” rel=”stylesheet” type=”text/css” />
<script type=”text/javascript” src=”<?php bloginfo(‘template_url’); ?>/javascript/imghover.js”> </script>
<link rel=”alternate” type=”application/rss+xml” title=”RSS 2.0″ href=”<?php bloginfo(‘rss2_url’); ?>” />
<link rel=”alternate” type=”text/xml” title=”RSS .92″ href=”<?php bloginfo(‘rss_url’); ?>” />

<?php wp_head(); ?>

Anybody can help?

DanoNH
(@danonh)

14 years, 7 months ago

I also couldn’t find the line

<?php wp_get_archives('type=yearly&format=link'); ?>

used in my theme. What I did find, and quite by accident I might add, was that my Privacy settings were off. See my blog post here for details.

-Dan