The simplest way is to to remove them with javascript. :p
Actually here is the way to do it with php.
You should extend Walker_Category
class.
Add this to you your functions.php
http://pastebin.com/MMkct4wW
Then call this extended class from your template this way:
$walker = new My_Walker_Category();
$args = array( 'walker' => $walker );
wp_list_categories($args);
Add any options you want to pass to the function wp_list_categories
to the array $args
.
Thanks, i actually did it with js, but this is probable better.
Thanks