pre_get_posts: Exclude categories except certain IDs

I’m trying to modify a query to exclude posts from “x” category, unless they have “y” Post ID’s.

I’m excluding the category using category__not_in. And I have my list of exceptional posts. But what I can’t figure out is how to use that list to override the category exclusion. post__in won’t do it; it presumes both should be true, and the result is no posts.

Does anyone know how this may be achieved, perhaps using $query->set? Or does this need to be done using an SQL query?

Thanks!