Programmatically Assign Templates From Plugin

Is there a way to programmatically assign which template a custom taxonomy should use? I have the following Custom Post Types, and Taxonomies, using the Genesis framework:

Dining (custom post type)

dining_types (custom taxonomy)
-restaurants (term)
-cafes (term)
-delis (term)
-street vendors (term)

Lodging (custom post type)

lodging_types (custom taxonomy)
-hotels (term)
-inns (term)
-bed and breakfasts (term)
-resorts (term)

Rather than create 8 templates, I could dynamically use the term name to serve related content (I have this working with a Page setup right now, but I’m switching to custom post types and taxonomies). Is there any way to simply tell all 8 of those terms to use “special-template.php” or whatever naming convention would be needed? The only detail is that they all need to use the same file, or at least 1 file per custom Taxonomy.

This could would preferably go in my site-specific plugin, or functions.php.