That's too bad. That's why I think refactoring is needed. If you implement it this way, I'll have to substitute my own cart that doesn't have all that overhead. The way you're designing it won't satisfy anyone in my opinion, and it increases load time for users that don't cache their pages, which seems unwise to me. While it's true that the impact is not as bad as reloading the page, it does reload all plugins, or am I missing something?
I don't know how much you follow online discussions about WordPress & plugins, but there's a vocal faction that criticizes WP plugins for consuming CPU resource. I'm currently testing Woocommerce on a site. With just one product defined, according to P3 (Plugin Performance Profiler), Woocommerce uses 50% of the CPU resources. With the vast majority of WP sites installed in shared hosting servers, CPU usage is critical. That's a primary reason for the popularity of cache plugins, not just to enhance speed, but to reduce CPU usage so accounts don't get suspended for excessive resource use. So whatever you do, it should at least do no harm, but loading wp-load.php an additional time for each page could be harmful in some cases.