Support » Plugin: FeedWordPress » Crisis: FWP makes new duplicates of posts every hour

  • Resolved energynet



    Every time Feedwordpress updates (once an hour) it makes a new duplicate of every single post that it had already created. I have a dozen sites linked to and am now getting 200 new duplcates an hour. I was sick for a few days before I caught this, and it literally posted nearly 10,000 articles onto the blog.

    For now, I set everything to pending mode so that I can delete the new posts. I have another site that has been working just fine for months. Am not sure if this is because I upgraded to WP 3.6 or not…

Viewing 13 replies - 1 through 13 (of 13 total)
  • – Was it working before 3.6 on this site? I saw this problem with wpdb::escape

    Did you try ‘FeedWordPress Duplicate Post Filter’ plugin? While not perfect you end up with less duplicates.

    Thanks for the response. I had to spend near 40 hours removing almost 10,000 duplicates! In answer, I hadn’t tried it on this site prior to 3.6. I have another site that has a number of mirrors working jut fine, so it could be 3.6 that is causing it.

    I was pretty desperate to sort this out and tried again. I deleted the FWP plugin completely several times, even rolled the site back to an old backup but was not able to purge the settings completely, so that no matter what I did, all of the settings would come back upon reinstallation and start doing the duplication process.

    There was another page that had some 16 posts I found about this here suggesting that people edit the syndication.feedword.php file in several locations when I did that, all categories were killed resulting in all posts on the site dying. I was able to bring fix that nightmare but certainly, that post had no answers.

    I’ve also sent several email messages to the Charles Johnson’s site and he has failed to respond whatsoever. It doesn’t seem to be the case that he’s actively monitoring either his FWP website or this support forum. I’ve seen this before where the programmer burns out, not having the energy to deal with all the followup for a well designed plugin. No grudges, mind you, just a bit saddened as I’d love to get it working, considering the fact that I tried 4 times spending half a week (full time just about) attempting to fix this.

    The only thing I haven’t done is uninstall all of the other plugins, but then doing that would mess up a bunch of stuff. Am considering doing it on a test site at some point… But by the sounds of it, your probably right in terms of what is going on…

    I believe FWP has a diagnostics switch which will output almost all decisions the plugin makes to your logfiles (nb -not- handy on a production site ). You can try to see if something interesting comes up.

    Otherwise it might be that the feeds you are using change something small every hour ( like a date, timer, etc ) which will cause FWP to see it as a new post ( posts should be identical otherwise they will be ‘new’) .

    It’s hard to advice anything more.

    I can imagina as plugin author it’s difficult to give fast support, especially on very complicated matters. It’s all for free and usually we (plugin developers ) have a boss or other work as well, which is paid so goes first.

    Good luck anyway

    Yeah, I can relate to the programmer’s situation. I used to do a bit myself 20 years ago. I kind of wished I’d kept up with some of the newer systems. I did set the diagnostics on, also played with a lot of settings as well. I had nearly 10 mirrored sites I wanted to do and it was doing it with all of them. About the only thing I didn’t do was strip all the other plugins out to see if there was a conflict.

    I had a similar problem with another plugin that worked really nice until they upgraded it, and then all hell broke lose. It took nearly 4 months for the guy to finally get around to doing a fix.

    I’m definitely not going to upgrade the other site to WP3.6 until I see this fixed as the mirror program is one of its most popular features, something that would be similar if I could get it to work.

    Will be patient, but can’t wait until it does work, would even motivate me to make a donation! 🙂

    Seeing the same issue on one of our sites. Coming in in the morning and having to delete 10,000 items in pending cuz of duplication. Not sure if the hash in the custom fields has anything to do with it or not but seeing the same hash over and over for duplicates.

    The plugin is also pulling very old items from sites that haven’t updated in a long time. I’m seeing items from March & April in my pending .

    Oh yeah.. had to turn on pending to keep them from doing live right on the site.

    Interesting comment about the Custom fields. Are you saying that you are getting a different #Code or the same syndication_item_hash #code for each duplicate? In my working version of WFP I get something like


    in that section of the Custom Fields settings. And yeah, my nightmare would also go all the way back to the mirror sites oldest post and make a new duplicate of it every hour.

    Did you per chance attempt to dismantle all of your other plugins to see if that solved this? I’ve got too many dang plugings, including at least a couple of which would mess up how posts look, so haven’t done this myself as of yet to see if this was potentially a cause.


    I’ve got it working for now.

    The trick is to turn off all plugins and start FWP first, then slowly reinstall your other plugins back one at a time.

    nice to hear, i’ve this issue also, i got duplicates posts before 1-2 months but then idk what happened and all worked well, now it’s back from itself and i even can’t get jetpack publicize work.. i don’t know what is the issue exactly but i will do what you did and see if that actually work.

    I’m definitely keeping a close eye on this, as its the last thing I want to do – delete 10,000 duplicate posts again… It took days to cleanup.

    I’d certainly make sure to get backups of everything, including every so often as you reinstall plugins. I was a bit shocked that nobody mentioned this as a possible solution, and spent more than a bit of time going back through the forum to look for a fix, even playing with the diagnostic tool, those advanced speed up settings, which were not exactly well documented as well as just about every other setting on the other syndication pages. Definitely not for beginners, but such an important tool. Am still awaiting comments on several other posts like how to deal with categories etc. too as some folks are more than a bit prolific with them, vs. using tags to fine tune larger subject areas.

    Good luck Too

    I’m having this problem in a major way, on a Network site. Most of the sites sort-of behave with the odd dupe, but 2 sites in particular have gone absolutely bonkers to the stage where I’ve had to write a script to delete dupes after FWP’s cron run while I investigate. I’m getting several dupes of each posts, often more than a dozen of each.

    The settings on all sites appear to be the same, however a colleague has noticed that these 2 sites seem to differ ever so slightly, with _synfresh appearing in a SELECT query, below. This doesn’t look like a valid query to me. (Note that SQL_CALC_ROWS is omitted by design, I hacked it out as the site doesn’t need pagination. This was long after the problem, so it’s unrelated.)

    SELECT _synfresh FROM wp_9_posts LEFT JOIN wp_9_postmeta ON [...]

    Additionally, I’m getting a shedload of errors in the error log, that seem to be generated by an errant query in FWP that’s trying to JOIN the same table to itself.

    [Wed Aug 21 00:01:17 2013] [error] [client] WordPress database error Not unique table/alias: 'wp_12_postmeta' for query SELECT wp_12_posts.ID, wp_12_posts.guid, wp_12_posts.post_title, wp_12_postmeta.meta_value FROM wp_12_posts INNER JOIN wp_12_postmeta ON (wp_12_posts.ID = wp_12_postmeta.post_id)LEFT JOIN wp_12_postmeta ON wp_12_posts.ID = wp_12_postmeta.post_id AND wp_12_postmeta.meta_key = 'fnp_featured_article' AND wp_12_postmeta.meta_value >= '1376780477' WHERE 1=1 AND (wp_12_postmeta.meta_key = '_feedwordpress_retire_me_15' AND wp_12_postmeta.meta_value = '1') AND (wp_12_posts.post_status <> 'fwpretired') GROUP BY wp_12_posts.ID ORDER BY wp_12_postmeta.meta_key DESC, wp_12_posts.post_modified DESC LIMIT 0, 10 made by require('wp-blog-header.php'), require_once('wp-load.php'), require_once('wp-config.php'), require_once('wp-settings.php'), do_action('init'), call_user_func_array, FeedWordPress->init, FeedWordPress->update_magic_url, FeedWordPress->update, do_action('feedwordpress_update_complete'), call_user_func_array, SyndicatedLink->process_retirements, WP_Query->__construct, WP_Query->query, WP_Query->get_posts

    /me crossed fingers someone from FWP hangs out here!

    Sorry, but you really should post a separate repair request for what’s happening to you. What solved my duplication problem as mentioned above, was to uninstall all of the plugins from the site, and then put FWP in first and then build the mirrored sites and carefully start putting back the old plugins as a test to make sure isn’t still happening. You could also do a repair on the database before rebuilding stuff…

    The guy that made this amazing plugin appears to be on vacation from doing support, which I’ve seen happen with other plugins as well. Good luck.

    For reference, it was NextGen Gallery. It was also responsible for the massive CPU overhead MySQL was suffering from. Always was a heap of crap of a plugin. <UNINSTALL>


    That would make sense, as the one site I had that’s worked perfectly didn’t have NextGen, while the other that failed did!

    At least for me.

Viewing 13 replies - 1 through 13 (of 13 total)
  • The topic ‘Crisis: FWP makes new duplicates of posts every hour’ is closed to new replies.