The same for me! After several reinstallations, I installed it manually and... The first time it opened a configuration window. I checked the «Override WP LaTeX» box (or sort of), after which the plugin displayed a message that I am using an obsolete version of MathJax (1.0.1) and I should configure the plugin. When I entered the config page again, it stopped displaying right after the first checkbox description THe rest of the config page did not load (neither did the «Save changes» button) (WP 3.4.1, too!).
Using header.php and configuring MathJax (to detect the [latex] opening syntax instead of \[ etc.) renders formulæ only if the post is opened in its own window. It does not render when several posts are displayed from the index page (which of cause may be caused by the InfiniteScroll plugin). I ask you to check if it is compatible with continuous page loading. I cannot check it due to the reason the plugin crashes the same way described by monsieur WebDrake.
Thank you for the further suggestions.