Change this:
<?php
if ( function_exists('register_sidebar') )
register_sidebar(array(
'before_widget' => '',
'after_widget' => '',
'before_title' => '<h4>',
'after_title' => '</h4>',
));
?>
to
<?php
if ( function_exists('register_sidebar') )
register_sidebar(array(
'before_widget' => '<li id="%1$s" class="widget %2$s">',
'after_widget' => '</li>',
'before_title' => '<h4>',
'after_title' => '</h4>',
));
?>
The %1$s will set the title of the widget as the list items id, using this id you can target specific h4 title tags and only hide those. For example
#archives h4{
diaplay:none;
}
Check the source code of your website in a browser after the page loads and you can see what all the id's were set to and then just use the css above to only hide h4 tags that fall within those id's like this:
#archives h4, #categories h4, #pages h4{
diaplay:none;
}
The above are all just examples. You could also just hide all the h4's like you did before and then only unhide certain ones
#sidebar h4{
diaplay:none;
}
#amazon h4{
diaplay:block;
}