custom post type with multiple terms display unique

So I have custom post types and custom terms. I have a list with projects to display and filtered under their terms on the project overview page. So if I have a post with multiple terms it will display it under all those terms.

so:

term1
project1, project2

term2
project1, project3

Now when I go to the detail page of the project it will display its term on top as a “category title”. However if it has multiple terms like “project1” when I click on the project1 under term2 it will display the title for term1, also the next and previous links arent working as wanted. I know a project is one object that contains 2 terms when you get the terms of that project. But how can I make it distinct so that it knows its a project under a selected term?