Lost in a Vocabulary Vortex

I am new to blogging. Understand widgets but do not understand their role or function.

Allows a user to add functionality to a sidebar without having to resort to coding.

What is the difference btwn a category, page, and tag and how do they work?

A page is static content. A category and a tag are two mechanism to ‘classify’ posts. Review WordPress Semantics

If I were to write a post, how do I get it to show up in the side bar?

Perfect example of using a widget–in Appearances->Widgets, use the Recent Posts widget. Or if you want something more sophisticated look at these widgets/plugins:
* http://wordpress.org/extend/plugins/query-posts/
* http://wordpress.org/extend/plugins/wordpress-loop/
* http://wordpress.org/extend/plugins/category-posts/
* http://wordpress.org/extend/plugins/list-category/
* http://wordpress.org/extend/plugins/bns-featured-category/