Order Tax name

Hey there!

I was wondering if anyone could help me solve this problem.
I’ve got the following part of my code:

$my_query = new WP_Query($args);
 if( $my_query->have_posts() ) {
 echo '<h2>'. $tax_term->name . '</h2>';
 while ($my_query->have_posts()) : $my_query->the_post(); ?>

With:

$args=array(
'post_type' => $post_type,
"$tax" => $tax_term->slug,
'post_status' => 'publish',
'posts_per_page' => -1,
'caller_get_posts'=> 1,
);

This part of the code outputs Taxonomy names, the rest of my code outputs the CPT’s connected to the relevant taxonomies. Which is irrelevant right now.

What I am trying to do is order the taxonomy names by ID.
However orderby => id does not work.

So yeah, anyone got an idea?

Thanks in advance!