Sort custom post type by taxonomy using multiple loops

I have the following code, graciously given to me by helgathevigking at stackexchange:

$terms = get_terms( 'genre' );

if( $terms ):

foreach( $terms as $term ){

    // The Query
    $args = array(
        'post_type' => 'book',
        'tax_query' => array(
            'taxonomy' => 'genre',
            'field' => 'term_id',
            'terms' => $term->term_id )
    );

    $the_query = new WP_Query( $args );

    // The Loop
    if ( $the_query->have_posts() ) {
        echo '<h2>' . ucwords( $term->name ) . '</h2>';
            echo '<ul>';
        while ( $the_query->have_posts() ) {
            $the_query->the_post();
            echo '<li>' . get_the_title() . '</li>';
        }
        echo '</ul>';
    }
}

endif;

/* Restore original Post Data */
wp_reset_postdata();

This code lists a loop for every taxonomy, so I can order the books (a custom post type) under their respective taxonomy. The problem is that this lists all of the posts under each taxonomy. so, there are several loops, all with every book despite the category.

Does anyone know how to modify this code to achieve only the posts in each taxonomy? (instead of all of them)