well, not an easy question, but an interesting one nonetheless 🙂

typically the majority of time spent in autoptimize is mainly in the actual minification of code that is not minified yet (purely based on filename; if the filename ends in .min.js or -min.js).

so generally speaking, the way to avoid this is;
1. have a page cache (you do) to avoid requests triggering autoptimize (as in that case the cached HTML will have links to cached CSS/JS in it)
2. for uncached pages; make sure AO can re-use previously cached CSS/ JS (from another page), in which case no minification needs to be done (for that you will almost always want to NOT aggregate inline JS, this almost always busts the cache)
3. for uncached CSS/ JS; make sure any minified file is recognizable as such in the filename (e.g. .min.css -min.js), this can lighten the minification-load considerably (I’ll add a filter in the next version of AO so you can tell AO a file is minified even if it does not have that in the name).

So based on this, some tips;
* make sure you’re not aggregating inline JS
* for your own code (CSS/ JS); make sure it is minified and that the filename confirms this (if you can convince the theme’s developer to do so, all the better (esp. the already minified but big wp-content/themes/bridge/js/plugins.js is a waste of precious resources)
* you could try switching to the legacy minifiers (see FAQ) to see if this improves performance
* you can also check if excluding some un-minified files from minification helps performance (e.g. that bridge/js/plugins.js)

hope this helps!
frank

excluding that pesky plugins.js file

try convincing the theme developer to rename it into plugins.min.js and the world will be a little bit better for it if (s)he does 😉