Filter custom content type by taxonomy w/correct template

This should be really simple but it escapes me.

I have a custom content type – portfolio
I created a taxonomy – ptype (Portfolio Types)

I already have a portfolio template that displays post of content type portfolio

Now I want to add a filter on the top of the page that will link to a specific set of post by category

ALL – WEB DESIGN – MARKETING – HOSTING (for example)

The slugs for the terms work for example if I use example.com/portfolio_category/web-design it loads the page but with the wrong template not the portfolio-template.

Can someone please tell me how to tie this all together?

Thanks in advance!