Product import and update issues

The standard product import and updater within woocommerce has an issue with completing the import or update file.

The store has 295,000 products, all imported.
Performance is hugely affected by the number of products, initally just over 15,000 imports in 1 hour. By the end, 5000 took 6 hours.
I can live with it.

But…

Despite giving a nice tick at the end of the import, the system has a nasty habit of missing the last 20 – 200 products in a 5000 product batch.

Thankfully it’s always the end of the file and the missed products can easily be copied to the beginning of the next batch and they import without an issue.

This I have replicated on a test site and the import behaves the same.

Additionally I’ve just done a price update on 93,000 products and this process has exactly the same issue.

Again by careful monitoring I can copy the missed lines and carry on.

Performance…

On the updates, I wasn’t expecting the process to be quite so slow.
2 hours to do 5000 – less the 121 that were missed at the end of the file.
It would take over 6 days to do the whole update.

I added an index & Unique to sku in wp_wc_product_meta_lookup, the whole update managed to run in just over 6 hours (less the 254 products missed off the end).

By comparison, the same update against my price master table (not in WP) which is keyed on the sku took 22 seconds on the same server. No products missed.