Duplicate posts since upgrade

  • koenvrancken

    (@koenvrancken)


    9 years, 10 months ago

    Allen,

    After all the actions as mentioned @http://wordpress.org/support/topic/duplicate-posts-since-update-to-2676-and-2675 I still got duplicates.

    I’ve build in some logging into the plug in (I must admit, I do not master the level of PHP as you do, so it might be incorrect).

    Please consider this logging for a rss file modified once a day (8AM):

    The plugin runs every hour.

    I can see the plugin searches for a key:

    SELECT post_id FROM ssft_postmeta WHERE meta_key = 'rssmi_source_link' and meta_value LIKE '%http://www.modellspur.de/?NL1402465502%'1
Array ( ) 1

    It doesn’t find the key, so it generates a post.

    The next 3 hours, the plugin finds the key, and does nothing. 

    SELECT post_id FROM ssft_postmeta WHERE meta_key = 'rssmi_source_link' and meta_value LIKE '%http://www.modellspur.de/?NL1402465502%'1

Array ( [0] => 4312 ) 1

    But on the fourth hour the search for the key fails and it generates a new post. And it does so for the next two times I manually trigger the plug in. And then it finds the key again and does not post a duplicate…

    Please provide me with information you need (logging, database queries) to solve this issue.

    — Although I think this topic is a copy of http://wordpress.org/support/topic/duplicate-posts-since-update-to-2676-and-2675 the Mod asked me to post a new topic in case I needed support —

    https://wordpress.org/plugins/wp-rss-multi-importer/

Viewing 15 replies - 1 through 15 (of 23 total)
1 2
  • Thread Starter koenvrancken

    (@koenvrancken)

    9 years, 10 months ago

    I did some more research, my – final – findings:

    1. the meta_value rssmi_source_link is not inserted into the database if the multi importer is run by the Cron job (manager) –> is causing the duplicates, because the search always fails.

    2. In the previous release the plugin also checked on duplicates by titles, but this code had been ‘deactived’ (comment // ) so therefore this check is no longer working.

    3. Some RSS feeds are not processed. I don’t know why. Is it because the limits of Number of Entries per Feed to Fetch and Number of Total Post Entries per Fetch are reached? (15 feeds, I set this to 100 / 800)

    4. the meta_value rssmi_source_link is inserted into the database if the multi importer is run manually (through the admin interface).

    1+2 Make the plugin to create duplicates. This mechanisme might by failing for a longer time, but in the past the duplicate check on the title prevented duplicate posts … IMHO … !!

    I removed the comments for the duplicate check on titles and will report on any (new) findings within a few days.

    Cheers,
    Koen

    Plugin Author Allen

    (@amweiss98)

    9 years, 10 months ago

    Hi Koen
    I had to take off the check for the titles because the php function was causing problems on some versions…if you want, try this file and see if it solves the duplicate problem.

    https://www.dropbox.com/s/slomcuvoue0dvzs/import_posts.php

    Thread Starter koenvrancken

    (@koenvrancken)

    9 years, 10 months ago

    Hi Allen, thanks for the update.
    I will upload the new import_posts.php today and let you know in a few days.

    btw: my own ‘hacking’ resolved the duplicate issue, but I’ve got this feeling that updates are missing now (did not really check on that, but I used to a have a 10-15 new posts a day, now only got 2), so I rather use your update!

    Thread Starter koenvrancken

    (@koenvrancken)

    9 years, 10 months ago

    Sorry, duplicates again…

    Examples of the rss feeds causings duplicates:

    modellspur_de.rss

    modellspur_nl.rss

    Do you want any additional logging ?

    Plugin Author Allen

    (@amweiss98)

    9 years, 10 months ago

    I can’t reproduce this problem…I’m using the most recent version of the plugin, and I get no duplicates no matter how hard I try.

    please read this to see how the plugin works to prevent duplicates..

    http://www.wprssimporter.com/faqs/im-getting-double-posts-with-the-feed-to-post/

    Thread Starter koenvrancken

    (@koenvrancken)

    9 years, 10 months ago

    Allen thanks for trying. I replaced your updated version with my own (where the comments for the title check are removed).

    You might update your faq, because it says:

    … and whether the title of the post is in the post table …

    And as far I know, the plug in does not do that any more (unless someone removes the comment-tags // 😉 )

    Plugin Author Allen

    (@amweiss98)

    9 years, 10 months ago

    ok..try this file and see if this helps…I put another check to see if there is a duplicate

    https://www.dropbox.com/s/slomcuvoue0dvzs/import_posts.php

    let me know if this works for you.

    Thread Starter koenvrancken

    (@koenvrancken)

    9 years, 10 months ago

    Allen, I installed the file and the import ran every hour (cron job). The link was not stored in the database (this is query I used):

    SELECT *
FROM  <code>ssft_postmeta</code>
WHERE meta_key =  "rssmi_source_link"
AND meta_value =  "http://www.modellspur.de/?DE1403417476"

    So I got duplicates (every hour).

    Then I started the import manually, and the rssmi_source_link is inserted into the database!

    I don’t know anything about the WP cron jobs, but could there be an issue with connecting to the database?

    Plugin Author Allen

    (@amweiss98)

    9 years, 10 months ago

    well, the cron file hits the overall function, that should have no different effect than clicking the fetch link (manually) since they do the exact same operation. Are you on a production server or on a development server (like running on your personal computer)?

    Plugin Author Allen

    (@amweiss98)

    9 years, 10 months ago

    ok..just tested this with just the cron and had no problem having the link stored in the database…again, not sure what kind of server you are on and how this might be affecting what is happening to you.

    Thread Starter koenvrancken

    (@koenvrancken)

    9 years, 10 months ago

    Hi Allen, I’m on a production server. Might (another plugin) something lock the table needed for registration of the rssmi_source_link ?

    My cron looks very clean – The feedtopost and multi event are the only ones running every hour.

    Jun 22, 2014 @ 17:46 (1403459204) Once Hourly wp_rss_multi_event
    Jun 22, 2014 @ 18:35 (1403462134) Once Hourly wp_rss_multi_event_feedtopost
    Jun 22, 2014 @ 19:34 (1403465640) Twice Daily wp_maybe_auto_update
    Jun 22, 2014 @ 23:19 (1403479184) Twice Daily wp_version_check
    Jun 22, 2014 @ 23:19 (1403479184) Twice Daily wp_update_plugins
    Jun 22, 2014 @ 23:19 (1403479184) Twice Daily wp_update_themes
    Jun 23, 2014 @ 4:55 (1403499305) Once Daily wp_scheduled_auto_draft_delete
    Jun 23, 2014 @ 6:59 (1403506762) Once Daily updraft_backup_database
    Jun 23, 2014 @ 11:24 (1403522667) Once Daily wp_scheduled_delete
    Jun 24, 2014 @ 6:59 (1403593162) Once Weekly updraft_backup

    At what point do you insert the rssmi_source_link ? Maybe I can add some additional logging to see if it fails (for whatever reason).

    Thanks.
    Koen

    Plugin Author Allen

    (@amweiss98)

    9 years, 10 months ago

    this is the line that inserts the post

    $post_id = wp_insert_post($post);

    yes, see if another plugin you are running is blocking my plugin from adding the post…I’ve heard the All in One SEO is messing up my plugin, but haven’t had a chance to test this out.

    Thread Starter koenvrancken

    (@koenvrancken)

    9 years, 10 months ago

    what does this line of code do?

    if(add_post_meta($post_id, 'rssmi_source_link', $thisLink)!=false)

    Is this the one that adds the meta tag for the rssmi_source_link (in the WP documentation on wp_insert_post I can’t find an update on the post_meta). So I assume that this is the line of code actually adding the post meta data? So in my case it fails (if executed through the cron … )

    Plugin Author Allen

    (@amweiss98)

    9 years, 10 months ago

    that line of code says this…..if adding the link to the post meta file does not fail, then go on to do more stuff..hmm, there is no obvious reason why it should fail just because cron triggered the function. I’ll need to see if somebody else has run into this problem (because I cannot reproduce this on my server where is clearly does not fail).

    Plugin Author Allen

    (@amweiss98)

    9 years, 10 months ago

    Here is something else….instead of using the cron, try hitting this page:

    http://{your url}/wp-cron.php?doing_cron

    and doing this will manually fire off the cron scheduler…something is going weird with the WP cron scheduler and I’m finding that hitting the scheduler manually solves the problem. If it does you can use an outside cron scheduler – read more about that here:

    http://www.wprssimporter.com/faqs/how-to-have-more-control-over-scheduling-of-feteching-feeds/

Viewing 15 replies - 1 through 15 (of 23 total)
1 2
  • The topic ‘Duplicate posts since upgrade’ is closed to new replies.