OK. I think I figured it out. The bug report includes a "patch" which is basically a way to hack the code. Here is what I did, use with discression since I don't know what I'm doing:
Open the file wp-commentsrss2.php in your root WP directory.
Find line #52, with this code:
<title>by: <?php comment_author_rss() ?></title>
Replace that with this:
<title><?php if ( (! is_single()) || (! is_page()) ) {
$title = get_the_title($comment->comment_post_ID);
$title = apply_filters('the_title', $title);
$title = apply_filters('the_title_rss', $title);
echo "Comment on $title";
} ?> by: <?php comment_author_rss() ?></title>
Hopefully that will do the trick!