How to show all entries with one tag/category ?

Change to the WordPress Default theme, add the Tag Cloud Widget and the Category Widget via Admin->Appearances->Widgets then visit your blog and click on some of the tag or category links to see how WordPress handles the display of archives.

Review Template Hierarchy to understand what Templates WordPress uses to render posts, then look at those Templates in the wp-content/themes/default folder to see how it uses the template tag, previous_posts_link() and next_posts_link(), to display navigation links.

Related plugin:
http://wordpress.org/extend/plugins/custom-post-limits/