Found the solution! You must enter twice (create a paragraph break) before and after the caption coding. Note -- I write my posts using the HTML view, not the "Visual" view.
I'd type a paragraph, hit enter twice, insert the image, hit enter twice, then type the rest.
Here's an example of how it will look in the HTML version:
Paragraph content, blah blah blah, whatever you want to say. It doesn't matter, just some random stuff. More blah blah blahs and such. Paragraph content, blah blah blah, whatever you want to say. It doesn't matter, just some random stuff. More blah blah blahs and such. Paragraph content, blah blah blah, whatever you want to say. It doesn't matter, just some random stuff. More blah blah blahs.
[caption id="attachment_124" align="alignright" width="350" caption="Caption content"]<img class="size-full wp-image-124" title="title goes here" src="http://www.image-source-url.com/image.jpg" alt="Alternate information" width="350" height="233" />[/caption]
Paragraph content, blah blah blah, whatever you want to say. It doesn't matter, just some random stuff. More blah blah blahs and such. Paragraph content, blah blah blah, whatever you want to say. It doesn't matter, just some random stuff. More blah blah blahs and such. Paragraph content, blah blah blah, whatever you want to say. It doesn't matter, just some random stuff. More blah blah blahs.
