I am searching for an answer to the very same/similar issue!
In my case, using the MediaElement.js to play an mp3 audio file within a lightbox (prettyPhoto variety). Haven't tried in the very latest Firefox, but so far it works so the audio does stop playing when the lightbox is closed in every browser I've tried EXCEPT IE (version 9, only one available to me right now). Shame to read newwer Firefox has the same issue for you...
OK, just now tested the Firefox on Windows I had (FF 13.0.1), and the audio player didn't even load or play, just an empty black rectangle in its place.
Firefox wants me to do an update, so I just did...its now at 14.0.1
Another test...and same thing. Empty black rectangle, no sound.
Unfortunately, I can't share a link. Private site, under development.
Also...redpockets, I see you just went for real window popups for the videos instead of lightboxes. And did you test your audio-in-lightboxes in IE? The player(s) in the lightbox(es) display, but no sound for me when clicking the play button, in IE9.