Some scripts (for example wp-comments-post.php) call wp_die if things go wrong. wp_die has HTML hard coded within it (!) - this is no good practice.
I think WP needs a more modular way of dealing with error pages - perhaps a control panel / plugin which lists all the possibile error message (HTTP '404', '500', plus WP-specific error codes like 'empty comment posted' etc) and allows you assign specific templates or hooks to them.