Support » Plugin: Query Monitor » Long loops result in eventual memory exhaustion

  • Resolved pikamander2

    (@pikamander2)


    I’m writing a function that imports a large CSV file, loops through it, and then calls update_post_meta a bunch of times per row.

    My function was dying halfway through because it was running out of memory.

    After calling memory_get_usage(false) a bunch of time, I discovered that the memory usage was growing after each call to update_post_meta, which was strange because it shouldn’t be sticking around in memory.

    Eventually after disabling all of the plugins on my WordPress installation and reactivating them one at a time, I narrowed the problem down to Query Monitor.

    In hindsight, I understand why Query Monitor increases the memory usage every time update_post_meta is called (because update_post_meta makes a query to the database and Query Monitor stores info about that query in the RAM).

    But still, it would be nice if there was some kind of warning, error, or prevention of this issue on Query Monitor’s part. In my case, the function was failing with a generic out of memory 500 error and no useful info in the error log. That made debugging the problem a real headache.

    Thoughts?

Viewing 1 replies (of 1 total)
  • Plugin Author John Blackbourn

    (@johnbillion)

    WordPress Core Developer

    Thanks for the message.

    Unfortunately there’s not much that QM can do in this situation because it’s got no way of knowing the context of the action being performed. In recent versions of QM I have been drastically reducing the amount of memory that QM uses when it stores information about queries, but it can only go down so much and eventually with enough queries you will exhaust the available memory.

    I would recommend either simply deactivating QM when you run a large import, or alternatively to refactor your importer so it operates in a more atomic manner, either via Ajax calls that chunk the import into batches or via a background job queue (coincidentally I’m writing exactly that for a client project as we speak) or via a WP-CLI command (QM doesn’t do anything when the site is interfaced via the CLI).

Viewing 1 replies (of 1 total)
  • You must be logged in to reply to this topic.