Stats in database and /wp-admin slowness

The Stats module doesn’t store any data in your own database, but it does indeed run some additional checks when you’re logged in, to make sure your own logged in visits are not recorded in stats. It also makes an extra call to the Stats API to display a Stats graph in your admin bar.

For most people, the performance hit isn’t really noticeable though. I’d recommend that you give it a try, and see how it goes!