[Plugin: WordPress.com Stats] Just a note: we don't count your own visits to your blog.?

The stats page has this line:
Just a note: we don’t count your own visits to your blog.

But I swear it does count my visits. If I am working on something and end up refreshing the page 100 times, sure enough there are suddenly a large amount of visits shown in the stats. It will go from something like 20 visits per day to 120 visits today, and tomorrow it will fall back down to normal levels.

So why are my stats tracking my own visits? How can I get it to stop? It pretty much defeats the purpose of tracking stats when the numbers are influenced by my visits and I can’t weed them out.

http://wordpress.org/extend/plugins/stats/