If the Visual Tab was selected on a previously viewed editor page, then the Visual Tab will be the selected view for a newly viewed editor page. If you view a page with the Visual Editor disabled, you still see the Visual tab first anyway. I'm sure overcoming this is a nightmare or maybe even impossible, but has it been attempted?
I've only tested on WP 3.5. I didn't test on 3.4.2.
I can't believe this crap isn't stock WP functionality anyway. It begs the common sense question "Why have a text editor at all if you're going to paralyze people who use it? Duh." Automattic doesn't seem to think the problem all the way through.