WordPress › Support » [Plugin: Relevanssi - A Better Search] Index keeps deleting itself and showing no results

spsmw
Member
Posted 3 months ago #

Lately when I search for something on my site I get no results, even when I know there should be. When I go to the Relevanssi settings, it says "Documents in the index: 1". I then build the index (so the documents in the index = 88 or 150 or whatever) and the search works for a while. But then I try a search an hour later, I get no results, and when I go to the Relevanssi settings it says again, "Documents in the index: 1." What gives?

Has anyone else had this problem?

For what it's worth, I don't have query_posts() in my search template.

http://wordpress.org/extend/plugins/relevanssi/
msaari
Member
Posted 3 months ago #

Somebody else was complaining about the same problem. Unfortunately I don't know what might be causing this. Which version of Relevanssi are you using?
spsmw
Member
Posted 3 months ago #

I'm using Version 2.9.12. Let me know what other information might be helpful?
alex.ciarlillo
Member
Posted 3 months ago #

I am having the same issue. Also version 2.9.12. Caching is disabled. Indexing 3-4 custom post types. I can provide more details if you let me know what would be useful.
msaari
Member
Posted 3 months ago #

When that happens, what's left in the index? What's in your Relevanssi database table?
spsmw
Member
Posted 3 months ago #

msaari, how do I check my Relevanssi database table?
spsmw
Member
Posted 3 months ago #

Or (more accurately), are there particular things you want me to look for? Right now the index is working right, so when I go to the wp_relevanssi table I see what appears to be all the important words from my articles. I'll bop over next time my index goes to 1 and see if looks different?
spsmw
Member
Posted 3 months ago #

Okay. Did some sleuthing. This morning my index said "1" so I went back to wp_relevanssi. It appears to only include the most recently created post (in this case with ID 1007). Which matches what the Relevanssi options page has for "Highest Post Indexed," incidentally.

(This is pretty typical when the index goes missing — only the most recent post/page/whatever can be searched until I rebuild the index.)

So somehow creating a new post doesn't ADD things to the table, it REPLACES the whole table.

I hit "Build Index" and looked back at my table and it's all full again, but I reckon as soon as we create a new post it'll be wiped except for that post's searchable terms.

Let me know if I can provide you with any other information.
msaari
Member
Posted 3 months ago #

Thanks, that was the information I was after. Unfortunately I don't really have a clue why it would do that, it shouldn't remove anything when adding a new document. Please let me know if that happens every time you add a new post.
alex.ciarlillo
Member
Posted 3 months ago #
Just got back to work on this site. Index was empty so I rebuilt: 333 documents in the index, highest post ID 7958. Viewing the table returns expected 18,682 rows. The results look pretty straightforward and accurate.

Add a new post. Documents in index: 1, highest post ID: 7979 (the ID of the new post). View the table: 1670 rows! all the document IDs are 7979, but I noticed a pattern:
```
1, 7979, 'uncategorized', 1, 4
2, 7979, 'ciarlill', 1, 5
3, 7979, 'ciarlill', 1, 1
4, 7979, 'test', 1, 1
5, 7979, 'test', 1, 0
```
This was repeated over and over. The post title was "ciarlill test" and the body was just "test" and it was categorized in "uncategorized". Not sure why those 5 rows would repeat 334 times.

Rebuild the index again -
Documents in the index: 334 (coincidence??)
Highest post ID indexed: 7979

Viewing the table I get 18,687 rows. (+5 from the last index build) Just to check:
```
SELECT * FROM<code>www2-wordpress</code>.<code>wp_relevanssi</code> WHERE doc = 7979
14618, 7979, 'uncategorized', 1, 4
14619, 7979, 'ciarlill', 1, 5
14620, 7979, 'ciarlill', 1, 1
14621, 7979, 'test', 1, 1
14622, 7979, 'test', 1, 0
```
5 rows as expected, not 1670. So I think it is safe to say that something very strange is happening when new content is added. Let me know what other tests I can do or info I can provide to help.
spsmw
Member
Posted 3 months ago #

To echo Alex, it does appear to happen every time a new post is created.
msaari
Member
Posted 3 months ago #

Alex, one thing you should try is to see what gets deleted when a post is added. Find the function relevanssi_remove_doc() and check out what happens with the MySQL query there. It should only delete the post in question, but it seems to delete everything for some reason.

Also, you might want to check how many times relevanssi_index_doc() is called when you create a post, drop a mail() call in there or something.
alex.ciarlillo
Member
Posted 3 months ago #
Okay some more info.

relevanssi_remove_doc query looks perfectly fine. It is getting the correct ID and being called at post creation on publish. The full creation of a post generates like 2-3 or so calls to this. Don't think the problem is here.

relevanssi_index_doc is doing something crazy. I tried dropping a mail call in 2 places:

Beginning of functon
```
2154 function relevanssi_index_doc($indexpost, $remove_first = false, $custom_fields = false, $bypassglobalpost = false) {
2155     mail('me@me.com', 'RELEV INDEX CALLED', '');
2156     global $wpdb, $relevanssi_table, $post;
2157     $post_was_null = false;
2158     $previous_post = NULL;
```
Also after getting the post
```
2190     // Finally fetch the post again by ID. Complicated, yes, but unless we do this, we might end
2191     // up indexing the post before the updates come in.
2192     $post = get_post($ID);
2193     mail('me@me.com', 'RELEV INDEX CALLED', $post->ID);
```
Both times this seems to get called HUNDREDS of times. I had to restart Apache and I'm still clearing out my inbox and getting delayed mail. I'm afraid to approach debugging it with this method again but will try some other steps. When I would click 'Add New' the page would load but was still busy and would remain so until I closed the tab. Somehow adding the mail() call caused a side effect. I'm not sure if this side effect caused the 100's of calls or if the 100's of calls to this function all processing my mail() lead to the page hanging. Like I said I am going to continue to investigate but this is a pretty complex function and a large functional piece of relavanssi itself.

I cannot get an exact count of how many calls yet, but I have a hunch it might match the number of items previously in the index. In my case 333.

Any other ideas of where I could investigate?
alex.ciarlillo
Member
Posted 3 months ago #
Okay I added error_logging to the first line of relevanssi_index_doc.
```
error_log('relevanssi_index_doc called');
```
Add New Post. Page did not hang (assuming because it is not processing a huge number of mail() calls).
```
ciarlill@staff:relevanssi$ grep 'relevanssi_index_doc called' ../../debug.log | wc
    333    1332   16983
```
So it appears that when addind a new post relevanssi_index_doc is being called for every post that *was* in the index. In my case 333 times.

EDIT: Also added a debug message to relevanssi_build_index. Looks like this is being called on 'Add New Post' also.
msaari
Member
Posted 3 months ago #

Ouch. That's certainly the cause of your problems.

Now the only question is why on earth relevanssi_build_index() is being called - it's not hooked anywhere and Relevanssi only calls the function when user requests the index being built...

Here's something you could try. Find these lines in the plugin (around line 2550):

if (isset($_REQUEST['index'])) {
if (isset($_REQUEST['index_extend'])) {

Change 'index' and 'index_extend' to something else, anything. That'll break Relevanssi indexing feature for a while (so make sure you have a good index before you do that), then try adding a new post.

I suppose there's a chance that for some reason something in your WP installation sets $_REQUEST['index'] for some reason (you might want to add some check to see if that's actually the case), and if that's the case, that might trigger Relevanssi indexing accidentally.

If it's that simple, then it's easy to fix (I'll just change those request ids to something unique).

Thanks a lot, it's great to have somebody doing something useful to sort this out, it's always frustrating when several people have a serious bug that I just can't reproduce myself.
alex.ciarlillo
Member
Posted 3 months ago #
Okay first thing I did when I got in this morning was basically as you suggested. Instead of changing the REQUEST keys to something else though I simply commented out the lines below and did some error_log output to see what that values actually were. Surprisingly NOTHING showed up. I had the same thought yesterday and briefly looked through net requests in Firebug and didn't see anything but did not have time to do the above. So now I'm even more perplexed. Then I figure it has to be one of my personal plugins. Disable them all, still no joy. THEN I said screw it and grepped my entire WP directory for references to relevanssi_build_index... and what did I find?
```
relevanssi/readme.txt: if (!wp_next_scheduled('relevanssi_build_index')) {
relevanssi/readme.txt:  wp_schedule_event( time(), 'daily', 'relevanssi_build_index' );
relevanssi/relevanssi.php:function relevanssi_build_index($extend = false) {
relevanssi/relevanssi.php:            relevanssi_build_index();
relevanssi/relevanssi.php:                      relevanssi_build_index(true);
role-scoper/admin/relevanssi-helper-admin_rs.php:               if ( function_exists( 'relevanssi_build_index' ) )
role-scoper/admin/relevanssi-helper-admin_rs.php:               relevanssi_build_index();
```
ROLE SCOPER!

Here is the offending file:
```
1 <?php
  2 class Relevanssi_Admin_Helper_RS {
  3     function rvi_reindex() {
  4         if ( function_exists( 'relevanssi_truncate_cache' ) )
  5             relevanssi_truncate_cache( true );
  6
  7         if ( function_exists( 'relevanssi_build_index' ) )
  8             add_action( 'shutdown', array( 'Relevanssi_Admin_Helper_RS', 'do_reindex' ) );
  9     }
 10
 11     function do_reindex() {
 12         ob_start();  // as of version 2.9.12, no way to rebuild index w/o outputting message
 13         relevanssi_build_index();
 14         ob_end_clean();
 15     }
 16 }
 17 ?>
```
So I have not yet figured out WHAT this is or WHY it is or how to disable it without hacking the code. But I just wanted to share my findings as soon as possible. I can't believe I did not think to do this before. Should have been one of the first things I did, but it appears to be a completely un-documented piece of Role Scoper. I'm sure it is there for good reason and maybe I just missed it in the FAQ or Docs for role scoper (will go back and check shortly).

Huzzah!

Always glad to help. You provide a great plugin and reporting on the debugging and such I have to do anyway is the least I can do.

EDIT: Just to be clear, commenting out these lines from the role scoper file will fix the issue:
```
7         if ( function_exists( 'relevanssi_build_index' ) )
8             add_action( 'shutdown', array( 'Relevanssi_Admin_Helper_RS', 'do_reindex' ) );
```
Looked into the role-scoper code a bit more. Every time an object is saved it looks for the existance of 'relvanssi_query', then calls rvi_index() which hooks relevanssi_build_index() to 'shutdown' (called every time PHP execution is about to end). That's roughly the chain of events leading the problem. Looked into the Role Scoper forums and found that this may be related to relevanssi indexing post excerpts and such which then allows for users to see content that Role Scoper would typically not allow since it is held in the index and not filtered through the usual search results. I'm not delving much further into the issue as we really use Role Scoper for editor roles and not public so this does not apply for us. I will continue to keep up with this thread and look forward to a solution that doesn't require me to remember to comment those lines out every update ;)
spsmw
Member
Posted 3 months ago #

This is all very helpful. We do use Role Scoper for visibility/privacy issues, so I may do some experimenting with commenting out that section and seeing if restricted pages then show up in search. Alex, stellar detective work.
msaari
Member
Posted 3 months ago #

THANKS A LOT! No surprise then that I couldn't reproduce the error.

I'll bring this to the attention of the Role-Scoper developer. I'm sure he has a reason for that code, but it can't work this way.
kevinB
Member
Posted 3 months ago #

Thanks Mikko, for contacting me about this.

Role Scoper 1.3.55 is now available to eliminate the excessive reindexing on post save.
bgajus
Member
Posted 3 months ago #

kevinB

I also noticed that Role Scoper even the most updated version (1.3.55) for some reason or another breaks my menus when a search is performed. Any idea what might be causing this?? Using most up-to-date versions of both Relevanssi and Role Scoper.
bgajus
Member
Posted 3 months ago #

let me follow up by mentioning that if I use Relevanssi version 2.9.12 the issue with the menus goes away.
outlierdesign
Member
Posted 2 months ago #

Thank you very much for figuring this out! Really saved me a lot of stress.