I was going to insert a single photo (thumbnail) into a blog post and that photo had a caption that had a quotation mark (").
Well, when I inserted it and checked out the preview the caption didn't appear (but the caption appeared fine with other photos).
Example:
[caption id="attachment_1127" align="alignleft" width="150" caption="That sign said "Do not pass""]
It would probably help to just change the quotation mark to "