[Plugin: Relevanssi – A Better Search] Stacking no-result:no-result:no-result … queries by Google

After posting here, I realized that this support-forum isn’t about Relevanssi, but about plugins in general. Sorry about that. But if Relevanssi isn’t the cause, then this is maybe a correct place to ask ?

So what is causing this ?
The WordPress setup we have is nothing special. Only a handfull of plugins. And our site is clearly not the only site that has this problem.

We have had the line:
Disallow: /search
in our robots.txt file for a long time. I guess that isn’t enough, and we need /?s= in there as well. But I rather see the root-cause disappear than just having every WordPress user in the world change his robots.txt file manually.

I figured out how google can pickup weird search-URLs. We are running google-analytics. When some broken (or weird) site is generating those searches with the nested no-result: queries, the resulting page will trigger google-analytics. And google will be notified about the existance of the no-result: page. Maybe google uses that information in their page-ranking algorithms ? Not sure if this is what happens, but it could explain one part of the puzzle.

Thanks for the suggestion. I’m an old C-programmer who used to write C-code for networking devices. I have no knowledge about php, and I’m not sure I wanna check out all WP code to see how it hangs together. I was hoping for a log-function of WP, where I can just go through all http-requests. Maybe I’ll see if I can write some code.

I’ve grepped through all the php-code. The only place where I could find the exact string “no-results:” was in the google analytics code.
From googleanalytics.php:

} else if ($wp_query->is_search) {
$pushstr = “‘_trackPageview’,'”.get_bloginfo(‘url’).”/?s=”;
if ($wp_query->found_posts == 0) {
$push[] = $pushstr.”no-results:”.rawurlencode($wp_query->query_vars[‘s’]).”&cat=no-results'”;
} else

It looks like the string “no-results:” is pre-pended to the search-string. This seems like a place where excessive no-results: could be prepended.

I disabled google analytics for a few minutes on our website, and I still saw new searches with the mangled query. 🙁 It’s very weird. I’ll look into it again this weekend.

The problem is happening at many sites.
When searching on google for “no-results:no-results:” I’m getting 15.8 millions results ! Although google only gave me 355 results. Still doesn’t look good. I’m surprised nobody ever looked at this before.

http://www.google.nl/search?complete=0&q=%22no-results%3Ano-results%3A%22

One more update.

I couldn’t believe something was wrong in the google-analytics code. Google isn’t that sloppy. But then I realized that that code does not come from Google. It is part of the “Google Analytics for Wordpres” plugin. And the code is written by a volunteer from the WP community, not by Google.

When I looked at a lot of those websites that had the same problem, I noticed they were all using the “Google Analytics for Wordpres” plugin. So this plugin might very well be the cause of the problems.

I disabled it, and replaced it with another plugin.
Ultimate Google Analytics
Let’s hope this fixes the root of the problem.
We should know in a few days. I’ll update this post.

So there’s three parts to the solution.
1) Use a different GA plugin.
2) Add rules to robots.txt to prevent googlebots crawling for old malformed search-URLs.
3) Wait for old search-URLs to depricate from the google database.

If this turns out to fix the problem, I wonder if I should notify parties involved. Would Google remove the bogus 18 million entries from their database ? Should I contact the author of the GA for WordPress plugin ?

Anyway, Relevanssi had nothing to do with this. It was a very useful tool to warn us that something was wrong. Thanks for all your effort, msaari !