Loading widgets conditionally to the widgets panel based on page template used?

I am trying to figure out a way to load widget areas in the “admin => widgets” area of the admin dashboard based upon the header, page or footer template that is currently loaded by the theme user.
In other words, I have an admin options panel and I have several footer templates, header templates and page templates that can be loaded by a drop down selector. When the user selects a template he or she can add widgets and they will display on that template.
Currently all of the templates have different widget areas and configurations. The problem is that when the theme is loaded, all of the widget areas display as normal. However this is undesirable due to the amount of widget areas I have and it makes a really long list of widget areas display all the time.
So what I am hoping for is to find a way to make only the widget areas load that are actually being used by the selected templates. In other words, how can I fix it so that when a user chooses “lets say” the 3 column footer or the 4 column footer or the header with the ad widget or whatever, only those widget areas will display in the admin widgets area and not all of the other ones? I hope this makes sense as its been making me crazy for days.