I've just upgraded to 2.6, and very nice it is to, thanks for the good work.
There is one thing that's irritating me, though - the new caption system uses the same text for the caption and the alt text. This seems pretty clearly wrong, to me. The caption is to provide information about the image, while the alt text is to provide alternative content _to_ the image; these are not conceptually the same. Imagine, say, a graph; the caption should say what the graph represents, and perhaps where the data comes from, etc ("WordPress awesomeness by time, based on data from the WordPress Awesomeness Institute"). The alt text, on the other hand, should summarize the information ("WordPress awesomeness has increased from 90% awesome in 2006, to 120% awesome in 2008").
More immediately, making the caption and the alt text the same is pointless - it means that, if the user is not viewing images, they will see the same information _twice_. It would be good to be able to specify separate text for each, so that one can provide useful information about the image for the caption, while providing an alternative to the image in the alt text.