infinite_scroll_query_args and pre_get_posts

I am attempting to get Infinite Scroll’s “Click” behaviour to work with an offset.

My code so far is based on various forum posts and is as follows:

function my_custom_offset( $query ) {
        global $wp_query;

	if ( is_admin() || ! $query->is_main_query() ) {
		return;
	}

        if ( $wp_query->query_vars['paged'] > 0 ) {
                return;
        }

	if ( $query->is_home() && $query->is_main_query() ) {
		$query->set( 'offset', 4 );
		return;
	}
}
add_action( 'pre_get_posts', 'my_custom_offset', 10 );

function my_jetpack_infinite_scroll_query_args( $args ) {
        if ( is_home() ) {
                $args['offset'] = null;
        }

	return $args;
}
add_filter( 'infinite_scroll_query_args', 'my_jetpack_infinite_scroll_query_args' );

My offset is being successfully incorporated, but when I reach the end of my posts, the “Older posts” button still shows when no further posts are to be loaded. Once, clicked, the “Older posts” button disappears.

When I remove my custom action and filter, the “Older posts” button does not appear when there are no more posts to load.

Any suggestions or tweaks to my code much appreciated. Thanks.