HTML trouble

I am finding something strange happening with the HTML, but since I am ignorant about HTML and usually use the Visual editor, I am not sure if it is a problem. Advice most appreciated. (I am using Weaver II theme):

FIRST:

In the html it says this. I will put (…) to abbreviate:
<p style=”text-align: left;”>Various text</p>

[caption id= (…) [/caption]

NEXT:
I sometimes change the text when I update my site. So I go into the Visual Editor box, and I see:

Various text

IMAGE

So, I delete “Various text” and I paste in “Alternate text”.
However, when I then check the HTML, it says:

Various text

[caption id= (…) [/caption]

When I Update, and then View Page, it looks fine. But I am worried that it might cause some problems, if not for me on Firefox, then maybe for someone else? Since the <p style=”text-align: left;”> and the </p> were there originally, even though when they are gone it does not seem to change anything when I view the page, I am assuming they are there for some reason?

I know I could just put them back in, but this is an ongoing issue. Because I much prefer to use the visual box for entering text. Am I doing something wrong? Or is there a fix?

Many thanks.