Title: Use this plugin cautiously!
Last modified: August 20, 2016

---

# Use this plugin cautiously!

 *  [dorianj](https://wordpress.org/support/users/dorianj/)
 * (@dorianj)
 * [13 years, 3 months ago](https://wordpress.org/support/topic/use-this-plugin-cautiously/)
 * I was using this plugin to scrape prices from amazon. Apparently every time the
   plugin is called, it downloads the entire webpage that you are scraping and stores
   it in the wp_options table. This would be fine if the plugin would delete those
   stored pages after a certain amount of time but it doesn’t! 🙁
 * MySQL kept crashing (and taking my dedicated server down with it) because the
   wp_options for my site grew to 1.3GB (yes, GIGABYTE) whereas this table should
   normally be less than 1MB. Basically, every time someone visited a page on my
   site, this plugin was adding a new row of ~300kb of data to the wp_options table.
   YIKES!
 * If you have a small site, then maybe this plugin would be okay, but for a larger
   site that needs to scrape data, this plugin in its current state is not the best
   solution.
 * [http://wordpress.org/extend/plugins/wp-web-scrapper/](http://wordpress.org/extend/plugins/wp-web-scrapper/)

Viewing 8 replies - 1 through 8 (of 8 total)

 *  [Sea Jay](https://wordpress.org/support/users/jcollier/)
 * (@jcollier)
 * [13 years ago](https://wordpress.org/support/topic/use-this-plugin-cautiously/#post-3414722)
 * Is this still correct with the current version?
 *  [reportica](https://wordpress.org/support/users/reportica/)
 * (@reportica)
 * [13 years ago](https://wordpress.org/support/topic/use-this-plugin-cautiously/#post-3414723)
 * If you set the Cache expiry, does it delete it from the database?
 *  Plugin Author [Akshay Raje](https://wordpress.org/support/users/akshay_raje/)
 * (@akshay_raje)
 * [13 years ago](https://wordpress.org/support/topic/use-this-plugin-cautiously/#post-3414725)
 * [@dorianj](https://wordpress.org/support/users/dorianj/) Thanks for bringing 
   this up. May be I should just revert to file based cache or possibly use the 
   native object cache class of WordPress to avoid all such issues.
 *  [square_eyes](https://wordpress.org/support/users/square_eyes/)
 * (@square_eyes)
 * [13 years ago](https://wordpress.org/support/topic/use-this-plugin-cautiously/#post-3414726)
 * So in the mean time how do we clear the current cache?
 *  [qlddrones](https://wordpress.org/support/users/websupportguy/)
 * (@websupportguy)
 * [12 years, 11 months ago](https://wordpress.org/support/topic/use-this-plugin-cautiously/#post-3414731)
 * I also would like to know the answer to this.
 *  [WebPixie](https://wordpress.org/support/users/webpixie/)
 * (@webpixie)
 * [12 years, 9 months ago](https://wordpress.org/support/topic/use-this-plugin-cautiously/#post-3414744)
 * Storing the info in a custom meta field might be a good answer.
 *  [badjesus](https://wordpress.org/support/users/badjesus/)
 * (@badjesus)
 * [12 years, 8 months ago](https://wordpress.org/support/topic/use-this-plugin-cautiously/#post-3414749)
 * Just set cron to remove transient files after x time period.
    However, the ability
   to instead store the data in custom fields would be HUGE!! This plugin would 
   be worth major dollars if you could store the data in a custom field.
 *  [Johana](https://wordpress.org/support/users/johana-m/)
 * (@johana-m)
 * [12 years, 7 months ago](https://wordpress.org/support/topic/use-this-plugin-cautiously/#post-3414763)
 * My hosting provider just suspended my account because of this, I’ve been looking
   all around for a solution and when I find it I can’t understand it :cry:!
 * [@badjesus](https://wordpress.org/support/users/badjesus/): Where are this files
   stored and how can I erase them?
 * Could you please explain me how can I do to empty all the old data daily or weekly?
 * All my sites are down until I fix this, they say is too “resource hungry” for
   a shared account.
 * Thanks!!!

Viewing 8 replies - 1 through 8 (of 8 total)

The topic ‘Use this plugin cautiously!’ is closed to new replies.

 * ![](https://s.w.org/plugins/geopattern-icon/wp-web-scrapper.svg)
 * [WP Web Scraper](https://wordpress.org/plugins/wp-web-scrapper/)
 * [Frequently Asked Questions](https://wordpress.org/plugins/wp-web-scrapper/#faq)
 * [Support Threads](https://wordpress.org/support/plugin/wp-web-scrapper/)
 * [Active Topics](https://wordpress.org/support/plugin/wp-web-scrapper/active/)
 * [Unresolved Topics](https://wordpress.org/support/plugin/wp-web-scrapper/unresolved/)
 * [Reviews](https://wordpress.org/support/plugin/wp-web-scrapper/reviews/)

 * 8 replies
 * 9 participants
 * Last reply from: [Johana](https://wordpress.org/support/users/johana-m/)
 * Last activity: [12 years, 7 months ago](https://wordpress.org/support/topic/use-this-plugin-cautiously/#post-3414763)
 * Status: not resolved