Support » Plugin: WP Web Scraper » [Plugin: WP Web Scraper] Cache override (A solution to cache not expiring)

  • I am migrating a static page to WordPress, and am using this plugin to pull in the static pages I haven’t yet converted (great job on this plugin, btw…saved me a lot of effort to get this off the ground). I had a problem where it looked like the cache wasnt expiring, and it looks like other folks have had the same issue, so I wanted to share my findings and a possible solution.


    I used the inline tag [wpws …] to link an external web page into a WordPress page, using the default cache settings. I then realized I had to change the linked page; I edited it, uploaded it to the server, then refreshed WordPress and saw the old page. I then tried to add a cache setting (first cache = 1, then cache = 0), and it still wouldn’t pull the new page. I resorted to another work around I read on this site (commenting out the cache code in the plugin’s function.php#wpws_remote_request) before wanting to understand the issue a bit more.


    The problem appears to be in how the cache is stored. When you pull a page using this plugin, the page content is stored as a WP Transient using the cache timeout value set at the time. If I then change the cache value, it does not invalidate the cache already stored; the new setting only applies to new page data. Given that the default cache timeout is 60 minutes pulling the page once with no cache setting stores that page was cached for 60 minutes, and changing the cache timeout param afterward has no effect.


    I think this behavior is correct, but I wanted a way to force a cache_override. To do this, I implemented a “cache_override” option. In wpws_get_content, I added:

    'cache_override' => false

    to the end of the $default_wpwsopt array, and added code to grab that option into $cache_args underneath similar code for the “cache” parameter:

    $cache_args['cache'] = $wpwsopt['cache'];
    $cache_args['cache_override'] = $wpwsopt['cache_override'];

    Then, in wpws_remote_request, I added

    'cache_override' => false

    to the $default_cache_args array, and changed the conditional to check that value:

    if ( false === ( $cache = get_transient($transient) ) ||
    $cache_args['cache'] == 0 || $cache_args['cache_override'] != false ) {

    To use, edit your page and add cache_override=true to the [wpws…] declaration, then reload your page. It will refresh the content from the external link. After the page has loaded, you can edit your WP page and remove that option to fall back to the default caching strategy.

Viewing 1 replies (of 1 total)
Viewing 1 replies (of 1 total)
  • The topic ‘[Plugin: WP Web Scraper] Cache override (A solution to cache not expiring)’ is closed to new replies.