Limit access to media based on category

Hi

Is it possible to limit user access to manage media based on a category?

I need to setup a Multi-Author site but restrict authors to not only their own posts and pages, but also media items.

However, as the authors will be working in teams there’s a need to allow teams to manage each other’s items. I thought a media (or shared) custom taxonomy could be used to tag files then limit access to these.

Is this possible, or is there another this can be done?