Excluding first 4 posts

<?php
$paged = (get_query_var('paged')) ? get_query_var('paged') : 1;
query_posts('offset=4&paged=' . $paged); ?>

Try removing <?php wp_reset_query(); ?>.

Does this template file contain more than 1 loop?

That should work but try:

<?php
$paged = (get_query_var('paged')) ? get_query_var('paged') : 1;
$args= array(
	'offset' => 4,
	'paged' => $paged
);
query_posts($args);
?>

Then the problem is elsewhere – either in the theme or within your plugins.