Try this. Firefox will get the atom feed, and MSIE will get RSS2. Put it *before* the WordPress block:
#sniff the user agent
RewriteCond %{HTTP_USER_AGENT} firefox [NC]
#make sure that they're not already requesting the right thing.
RewriteCond %{REQUEST_FILENAME} !/feed/atom [NC]
#Rewrite it and start over to interpret the new URI.
RewriteRule ^(.*)/feed(/.*)?$ $1/feed/atom/ [N]
#sniff the UA string
RewriteCond %{HTTP_USER_AGENT} msie [NC]
#check that they're not already looking for the right thing.
RewriteCond %{REQUEST_FILENAME} !/feed/rss2 [NC]
#rewrite and start over.
RewriteRule ^(.*)/feed(/.*)?$ $1/feed/rss2/ [N]
Note that this won't work prior to Apache version 2, due to the greedy regex in the RewriteRule. However, since "feed" can be appended to so many different URIs, it'd be really tricky to solve the problem with mod_rewrite in an Apache 1.3 compatible way, and depend a lot on the specifics of your site setup.