Limit pagination by category?

Hey
The structure on my blog is this: I’m using categories to organize the content, so the menu bar is a list of categories, and when you click on each, you get the category page with one post displayed at a time. I added pagination (using previous_post() and next_post() in content-single.php) to navigate thought the posts.
The problem is that the pagination keeps going through all the posts, regardless of category. I need it to only show the posts in whatever category page I’m on. Any way to limit it like that?
Hope that’s clear!
Thanks!