This feature is already built into WP. Go into index.php and find the call to wp_list_cats.
It probably doesn't have any options right now, so add the following options so that the call now looks like:
wp_list_cats('sort_column=name&optioncount=1&feed=RSS');
The first option sorts the categories by name.
The second option will list the number of posts in each category.
The third option will give you the RSS feed for that category.
If you don't want to list the RSS feed but just use it, then you can simply point your browser to <site.com>/archives/category/<category>/feed and you will get the feed. This, of course, assumes you have permalinks enabled.
Regards