WordPress.org

Ready to get started?Download WordPress

Forums

Nginx
[resolved] Feature Suggestion for Additional Purge Options (26 posts)

  1. Darren Slatten
    Member
    Plugin Contributor

    Posted 1 year ago #

    It would be great if the plugin could incorporate these options in a future release:

    Nginx Settings

    • Plugin Options
      • ...
    • Purging Options
      • Purge Homepage:
        • ...
      • Purge Post/Page/Custom Post Type:
        • ...
      • Purge Archives:
        • ...
      • Purge Everything:
        • when a new theme is activated.
        • when the current theme is modified (e.g., customize header image, background, colors, etc.).
        • when a menu is modified or added.
        • when a widget is modified or added.

    An alternative might be to simply add a manual Purge Cache button to the settings page.

    Also, I want to say thank you for creating this plugin and making it publically available. :)

    http://wordpress.org/extend/plugins/nginx-helper/

  2. Saurabh Shukla
    Member
    Plugin Author

    Posted 1 year ago #

    Hi Darren,

    The 'purge everything' functionality is the pot of gold we also want to reach. (all our sites, clients' sites run on nginx and we built nginx-helper primarily as our own dog food) Unfortunately, there's nothing (yet) in nginx that can purge the whole cache at one go. So, the only way this would be done is, loop through all the urls whether they are cached or not and purge them.

    A possible optimisation is storing a list of urls that have been cached into the database. However, the sheer inelegance of these solutions have kept us back from adding a 'purge all' option or functionality. We've been looking for a more elegant solution, probably, from the nginx community. Or probably a brainwave that'll give us a nicer way to do it.

    If you can think of a good approach, please help us out with it. We'll be always obliged. Thanks for the encouragement, though.

    Regards.

  3. Darren Slatten
    Member
    Plugin Contributor

    Posted 1 year ago #

    Correct me if I'm wrong, but it looks to me like the plugin is already using that approach (looping through all the URLs whether they are cached or not), it just limits itself to certain classes of URLs (e.g., home, archives, posts, pages, etc.), depending on the WP action that initiated the purge.

    For the Purge Everything button, I was envisioning a simple function that calls each of the purge functions (or methods, rather) in succession. In fact, when I was looking at purger.php, I noticed there's a method that seems to do exactly what I was hoping for:

    private function _purge_them_all() {
        $this->log( __( "LET'S PURGE ALL THE BLOG.", "rt_wp_nginx_helper" ) );
        $this->_purge_homepage();
        $this->_purge_personal_urls();
        $this->_purge_all_posts();
        $this->_purge_all_taxonomies();
        $this->_purge_all_date_archives();
        return true;
    }

    But I couldn't find _purge_them_all mentioned anywhere else in the code. So...I guess my question is: besides the inelegance aspect, is there another reason why this wasn't made available to users? For example, does it cause significant performance issues at a certain point?

    I've only recently started using the fastcgi_cache directives, but as far as I know, there's no way to define separate cache paths for each server block. Right now, I can easily empty the cache because I only have one domain using it, so I just delete everything in the folder (via command line). But as I start to configure more domains to use the cache, it won't be that simple (unless I just delete the cache for all domains every time one of them changes). The way I see it, using something like _purge_them_all might be the only solution available to me.

    Regarding ideas for a better approach, nothing really comes to mind. The only alternative I can think of would be to loop through the cache directories/files themselves, read the content, parse out the KEY: httpGETwww.example.com/example-uri strings, and build a list of cached URLs, which you could then send to /purge.

    One problem with reading the cache directly is that the files are owned by the server, so I'm not even sure if PHP can read them on the average nginx setup, without users changing the permissions first. Even if they could, I don't see how this would be any better/faster than blindly sending all WP URLs to /purge.

    Yeah, nevermind that idea. I just talked myself out of it. LOL.

    Anyway, please let me know what's up with _purge_them_all, because I'm really tempted to hook it into something.

    Thanks.

  4. Saurabh Shukla
    Member
    Plugin Author

    Posted 1 year ago #

    I agree. what we are doing currently is also inelegant, as it stands, but the number of urls to purge when each such class is purged is incomparable to the number that get purged if all of them are called at the same time.

    Because there is no other way, the _purge_them_all function was written. We tested it too and then decided against it. We eat our own dog food, and this part of it was very tasteless!

    In any case, if you can try your hand at it, why not fork the project on [github](https://github.com/rtCamp/nginx-helper/tree/1.6.5), make the changes you'd like to and send a pull request. Kindly use this tag instead of the master. It already has a merged request that needs work upon. Do let me know if you need any help with that,

    In fact, currently, we are rather busy with our [BuddyPress Media plugin](http://wordpress.org/extend/plugins/buddypress-media/) and hence the next release of this plugin could be a little late, unless you are willing to contribute to it :). We are only doing bug fixes for a while.

    In case you buy into my request, just add the option to disable the 'purge everything' functionality.

    Regards.

  5. Darren Slatten
    Member
    Plugin Contributor

    Posted 1 year ago #

    Okay, I'll take a stab at it. I'll probably just go with the manual button approach, to reduce the likelihood of users automatically triggering it without understanding the impact.

    I do have one question though...

    In your recommended site configuration, you disable the caching of /feed/ URLs:

    # Don't cache uris containing the following segments
    if ($request_uri ~* "(/wp-admin/|/xmlrpc.php|/wp-(app|cron|login|register|mail).php|wp-.*.php|/feed/|index.php|wp-comments-popup.php|wp-links-opml.php|wp-locations.php|sitemap(_index)?.xml|[a-z0-9_-]+-sitemap([0-9]+)?.xml)") {
        set $no_cache 1;
    }

    But then I noticed in my nginx-helper/nginx.log file that the /feed/ variation of all URLs is also sent to /purge/. Seems to me if no one is caching /feed/ files to begin with, then I'd be able to remove them from the purge process, and thus reduce the number of uncached URLs sent to /purge/ by as much as 50%.

    So the question is: should I disable the purging of /feed/ URLs or leave it alone?

  6. Saurabh Shukla
    Member
    Plugin Author

    Posted 1 year ago #

    Hi,

    Now, that's an oversight. Feel free to make any such modifications. In fact, while doing that, if you find more, do what you feel is fine. If you could send in a pull request at each such change, we can discuss it over at github before merging, if there is any doubt.

    Only, leave the plugin headers and readme.txt as it is. We like writing changelogs and such metacode!

    Regards.

  7. Darren Slatten
    Member
    Plugin Contributor

    Posted 1 year ago #

    Okay, I just sent a pull request for the "Purge Everything" functionality. I didn't make any changes to the /feed/ links (yet). The only other minor change I made was to add a space between the top checkboxes and their labels (on the settings page). The nonconformity was killing me. ;)

  8. Saurabh Shukla
    Member
    Plugin Author

    Posted 1 year ago #

    Hi,

    I made a mistake by accepting a pull request on the master branch the last time and hence the repo went bad. That's why I had asked you to use the 1.6.5 tag to fork and send pull requests to. Your pull request, hence, contains code from the other pull request that we can't use right now.

    Now, I've turned things around a little and got the latest code onto the master. I can manually add your code (it isn't much, thankfully) but then your contribution will become invisible on github, which is not good.

    So, with apologies, I have to ask you to re-fork and resend the pull request, please!

    Thanks.

  9. Darren Slatten
    Member
    Plugin Contributor

    Posted 1 year ago #

    That's weird. I went to the URL you provided for tag 1.6.5 and forked it. If it didn't work, then...I guess I don't know how to fork a repo according to a specific tag.

    But if I understand you correctly, you made adjustments so that I don't need to use the tag. In other words, I'll just repeat the same process I used last time...but this time it should work.

  10. Saurabh Shukla
    Member
    Plugin Author

    Posted 1 year ago #

    Yes, just repeat that and it'll work.

    Regards

  11. Saurabh Shukla
    Member
    Plugin Author

    Posted 1 year ago #

    Hi,'

    I have added the code with some cosmetic modifications and released a new version. However, the contributors' list doesn't link to your profile.

    Could you give me your wordpress.org username that you use for logging in. It doesn't seem to be darren-slatten. Would it be Darren-Slatten? Because wordpress is case sensitive, as well!

    Thanks for the contribution.

    Regards.

  12. Darren Slatten
    Member
    Plugin Contributor

    Posted 1 year ago #

    The username I enter to log in is Darren Slatten, but in the wordpress.org URL it appears as /darren-slatten/. I'm assuming you figured it out though, because I updated to the latest version and the changelog link to my profile is working. Thanks for the recognition.

  13. Saurabh Shukla
    Member
    Plugin Author

    Posted 1 year ago #

    The changelog was fine, but i couldn't link your profile to the contributor's list. Done, now.

    Congratulations on your first plugin ;)

  14. pjv
    Member
    Plugin Contributor

    Posted 1 year ago #

    probably this is lack of php script knowledge, but is there no way to execute something like rm -r * at the os level in the cache directory?

    i don't know php at all, but in a python script you would be able to do something like that pretty easily.

  15. Daan Kortenbach
    Member
    Plugin Contributor

    Posted 1 year ago #

    Being able to do rm -r * from a button on a website is probably not a very smart move, but we could get in contact with the nginx developers and to discuss our needs.

  16. pjv
    Member
    Plugin Contributor

    Posted 1 year ago #

    yeah, i understand the issue, but if there were some way to confirm that the command was only running on the cache directory... guess that could be difficult to do.

    maybe force the user to enter the full path to the cache and also get the path to the nginx config from the user and then parse it to confirm that it has the same path entered for fastcgi_cache_path and then don't show the button unless the user checks a box that says they understand the danger and know what they are doing.

    probably still pretty dangerous and not worth people yelling at you.

  17. pjv
    Member
    Plugin Contributor

    Posted 1 year ago #

    fwiw, there's a lot of protective code in the function, but WP Super Cache pretty much does this.

    see the function prune_super_cache() in the file wp-cache-phase2.php

  18. Darren Slatten
    Member
    Plugin Contributor

    Posted 1 year ago #

    Plugins like WP Super Cache create the cache files in directories (e.g., /wp-content/cache) that are owned by the user executing the PHP script, so it's "safe" to allow PHP to delete those cache files.

    But Nginx Helper (i.e., the user executing the PHP script) doesn't actually create/own the cache files; the nginx web server itself does. Nginx Helper is more like an interface to nginx's native caching functionality. That is, it can trigger the purging of cache files, but it doesn't delete them directly.

    In fact, Nginx Helper shouldn't even have the necessary user privileges to delete the nginx cache directly, especially in a virtual hosting setup. This is because the cache files are written by nginx to a directory outside of the virtual host's document root. Allowing Nginx Helper to delete these files would be analogous to a WP plugin that could delete other websites on the same server--it's possible, but only if the server admin disregards basic security best practices.

  19. pjv
    Member
    Plugin Contributor

    Posted 1 year ago #

    got it.

    thanks for taking the time to write that explanation up.

  20. pjv
    Member
    Plugin Contributor

    Posted 11 months ago #

    looks like the plugin authors changed their minds about this issue and decided to go ahead and implement a simple unlink on the cache directory.

  21. rahul286
    Member
    Plugin Author

    Posted 11 months ago #

    Actually we are not unlinking cache directory itself but content inside it.

    This is easiest and only working workaround we found.

    If you have any suggestions, please pass on.

  22. pjv
    Member
    Plugin Contributor

    Posted 11 months ago #

    oops, yes, sorry for my sloppy language. what you said is what i actually meant.

    that was actually my original suggestion from a while back on how to purge the cache. it is fast and easy. it has the security issues that other contributors to this thread have brought up, but to me it still seems like a good solution.

    what i am struggling with now is a way to manage my nginx configs and my cache paths so that i can run multiple wordpress installs in parallel and be able to purge them independently.

    up to now, i have been storing the cache for all the sites in one cache directory which lets me set up a single fastcgi_cache set of directives in my http block of the main nginx.conf. since the cache key includes the hostname, it works fine in practice. the problem is that with this cleaner and simpler method of purging, any "full purge" is really a FULL purge - eliminating the entire cache for all the sites and that's not what i really want.

    so i am trying to come up with a clean scheme now to have separate cache directories for each site but not have to also write a full set of separate fastcgi_cache directives into the config for each separate site since the only thing that varies between them would be one element in the cache path and the name for the keys_zone. would be nice if there was a simple way to plop in a variable, but due to scope for variables and the fastcgi_cache directives in nginx configs, i can't figure out how to do it.

    anyone have any ideas or a working example?

  23. rahul286
    Member
    Plugin Author

    Posted 11 months ago #

    @pjv

    I am aware of this issue. We already have nginx-cache shared between many WordPress-sites.

    One idea would be to define multiple zones like:

    fastcgi_cache_path /var/run/nginx-cache/site1/ levels=1:2 keys_zone=WPSITEONE:1
    000m inactive=60m;
    fastcgi_cache_path /var/run/nginx-cache/site2/ levels=1:2 keys_zone=WPSITETWO:1
    000m inactive=60m;

    Then for site1, in wp-config.php you can add a line like:

    define('RT_WP_NGINX_HELPER_CACHE_PATH','/var/run/nginx-cache/site1/');

    Similarly, for site2, in wp-config.php you can add a line like:

    define('RT_WP_NGINX_HELPER_CACHE_PATH','/var/run/nginx-cache/site2/');

    Reason for not giving UI to update cache location is security concerns. Though at this point, I am not sure how much insecure it would be.

  24. rahul286
    Member
    Plugin Author

    Posted 11 months ago #

    Update:

    If you decide to take above path, please don't forget to update KEY_ZONE name in fastcgi_cache and fastcgi_cache_purge settings.

  25. pjv
    Member
    Plugin Contributor

    Posted 11 months ago #

    thanks @rahul286, i did not know that nginx let you configure multiple zones like that. it doesn't exactly give me what i want since i still have to fully configure every host separately, but at least it would make separate cache directories possible.

    i have just been trying (failing) to set up something like this (in http):

    fastcgi_cache_path /home/sites/cache/$host levels=1:2 keys_zone=$host:500m inactive=60m;

    but for this purpose $host is not substituted in this context (even though it is for other directives in the http block - why?), and i end up with a cache directory actually named "$host".

    now i'm wondering if there is a way to use the map directive to do something like this.

  26. rahul286
    Member
    Plugin Author

    Posted 11 months ago #

    I think $host is available inside server{} block and fastcgi_cache_path need to specified outside server{} block.

    Apart from this, I am not sure if all nginx directives supports variable in their all values.

    By the way, for using variable is stings, try a syntax like:

    /home/sites/cache/${host}

    Note curly braces around word/variable host.

Topic Closed

This topic has been closed to new replies.

About this Plugin

About this Topic

Tags

No tags yet.