The categories section in my admin area says:
Deleting a category does not delete posts from that category, it will just set them back to the default category University.
My default category is another one though. The file wp-admin/categories.php uses get_catname(1) to get the category, although 1 might not necessarily be my default category.
I'm not sure if this is only a visual bug or if this is also wrong when I delete a category. Should be easy to fix though. :o