I am experiencing the same problem (not sure if this is the place to “ditto” or if I should open a new thread). I’ve turned on the alternative link dialog setting to, in effect, disable the new WP inline linking dialog. It’s working well everywhere else, but attempting to add a link to an image caption results in the same issue that leehankyeol reported — the originally highlighted anchor text stays in the caption, but a new link is created in the body text of the post rather than within the caption tags. I can switch to HTML and copy and paste the code for the link into caption tag, but it would be wonderful to have this work without having to manually move it. And yes, the captions link properly when the Alternative Link Dialog is turned off.
Any ideas on a solution?