In summary, you could create a category.php for your active theme (if one doesn’t already exist), and in this edit down The Loop to display just the title of each post.
Then to display *all* posts in a category (while retaining whatever your posts-per-page setting is elsewhere), I’d suggest this plugin: