The featured post seems to be the latest post in that category, so if the original still hasn't scrolled off the front page, then it will appear in both the sidebar and the column. You can hide the featured post from the front page sidebar using the visibility settings.
I would love to be able to select a different or random featured post, as the latest post in category is not necessarily the one I want to feature. Does anyone know if this can be done?
EDIT: I just found a simple way to do this by creating a special "featured" subcategory and adding that category to the post I want to feature. This gives me manual control over the featured post.