You're probably testing with IE 6.0 or earlier, which is well-known to have a poor implementation of CSS--the language that controls your formatting.
There are two fixes for this: (1) get IE 7.0, or (2) modify the heck out of your style.css file in your theme package. I know how to do this because I'm a web dweeb, but the process is tedious and clutters your CSS.
Essentially, the older Microsoft browsers calculate height and width differently (wrongly) and ignore CSS rules for positioning (such as float).
Sorry, there's no easy way out, but to your credit, it's nothing you've done wrong. If you want to accommodate readers who have the old browsers, I suggest you post a caveat in your blog that they can upgrade either to IE 7.0 or FireFox at no expense.
We dumped IE 5.5 six years ago and 7.0 has been out for two years now. Personally, I'm out of patience with people who cling to past obsolete technology.