Why not? "Semantic elegance." Ps define paragraphs, so paragraphs should be put in Ps. A list is a list, a paragraph is a paragraph, etc, etc.
TheJason hit it right on the head. P tags mean something, and they should be used whenever that meaning applies. DIV tags don't mean anything -that's the point of DIV tags- and so they should be used only when a more suitable element (one with meaning) is not available.
It's sad. Back when people wrote by hand -or even with typewriters- they thought about what things meant when they wrote. Nowadays, all they care about is what it looks like, without realizing that you can get both meaning and looks at the same time. All you have to do is think about meaning first, and then deal with looks.
Ahem. Either way, what dogfood wants to do could probably be provided via a quicktag; perhaps something mimicking the old FIG tag that was proposed for HTML some years ago? Something like the following:
Translate this in a quicktag to a DIV tag floated and padded appropriately. This would contain a block-style IMG tag with the src provided, and the caption above or below that. Another way to do this would be with a two-row table, one row containing the image and the other row containing the caption, but the semantics of this are dubious (though not completely invalid; a figure and its caption could be said to be a form of tabular data even though it doesn't quite follow the typical HTML model of a table).