Ok, I kinda got it, and here's the code I'm using (copied from various sources plus things of my own)...
<?php
$months = $wpdb->get_results("SELECT DISTINCT MONTH(post_date) AS month , YEAR(post_date) AS year
FROM $wpdb->posts
WHERE post_status = 'publish' and post_date <= now( ) and post_type = 'post' GROUP BY month, year ORDER BY post_date DESC");
$posts = $wpdb->get_results("SELECT id, post_title, MONTH(post_date) AS month , YEAR(post_date) AS year
FROM $wpdb->posts wposts, $wpdb->terms wterms, $wpdb->term_relationships wterm_relationships, $wpdb->term_taxonomy wterm_taxonomy
WHERE wterm_relationships.object_id = wposts.ID
AND wterm_relationships.term_taxonomy_id = wterm_taxonomy.term_taxonomy_id
AND wterms.term_id = wterm_taxonomy.term_id
AND wterm_taxonomy.taxonomy = 'post_tag'
AND wterms.name = 'Reviews'
AND wposts.post_status = 'publish'
AND wposts.post_type = 'post'
ORDER BY post_date DESC");
foreach($months as $this_month){ ?>
<p><?php echo date("F", mktime(0, 0, 0, $this_month->month, 1, $this_month->year)); ?> <?php echo $this_month->year; ?></p>
<ul>
<?php for ($i = 0; $i <= count($posts); $i++){?>
<?php if(($this_month->year == $posts[$i]->year)&&($this_month->month == $posts[$i]->month)){?>
<li><a href="<?php echo get_permalink($posts[$i]->id); ?>"><?php echo $posts[$i]->post_title; ?></a></li>
<?php } ?>
<?php } ?>
</ul>
<?php } ?>
It's showing up properly, however I'd like it to NOT show month names if there are no posts that month, which I can't seem to figure out.
Any ideas?