I figured the issue I am having is somewhat related to the image problems other people are having, but since I could not find the exact same issue here, I decided to post it.
When I add an image to a post, using the same old way as the previous versions, and this image has a caption, WP automatically creates a
[caption] "shortcode" with the image link and the caption text.
In My 3.5-beta1 the visual editor is creating a line break between [caption] and the link element --
<a (...)>. I can remove it in the text editor, but if go back to the visual editor this line break is there again, causing the element
<div id='attachment_##' class='wp-caption'> to not be displayed properly.
As I said, it is probably related to other media issues I have seen around here, but since I haven't seen this exact same issue, I decided to post it.