Ah, OK. I misinterpreted the “automatically add the caption to the featured image” option to mean it would pull in the caption data automatically. I’m not entirely sure what will happen when checked because the only place I can think of a featured image appearing is atop the post (unless this also captions thumbnail views of the image in certain theme blocks or something).
