I confirm the performance degradation.
With “P3 Plugin Profiler” I could notice 0.87 second just for JetPack.
I’m already using W3 Super Cache plugin.
Any suggestion?
If you use different Jetpack modules, each one of them will indeed have an effect on your site’s loading speed, much like different plugins would.
If you use the Stats module, it’s also worth noting that the module will run some additional queries to get information about Stats on every page load when you’re logged in to your admin account. Your readers, however, won’t trigger this extra query.
I would consequently recommend that you run the P3 plugin in a different browser or in an incognito window, where you won’t be logged in.
I hope this helps.