This happens in WP3.6 and WP3.5.2 and maybe earlier versions as well (I didn't have the means to test). Also, I'm not sure as of which version of the plugin this happens.
Simply clicking around the WordPress backend, from section to section, page to page, etc. takes significantly longer (one full second or more) with the plugin active, as opposed to just fractions of a second when it's inactive. I tried with clean WP installs, default theme, etc. and narrowed it down to this plugin.
You would think that, theoretically, the interface should actually be snappier or at least stay the same (since there are no update checks) but the exact opposite happens!