I discovered that the plugin doesn't strip the caption shortcode that was added in 2.6, so I found out how to fix it. Here's how:
How to remove the caption shortcode.
Comes down to adding
$text = preg_replace( '|\[(.+?)\](.+?\[/\\1\])?|s', '', $text ); at the right place in the plugin.