If you have 2 taxonomies that use the same term, then the groups will use the first taxonomy (possibly alphabetically) instead of remembering which of the taxonomies it was actually selected in.
Steps to reproduce:
1) create a category named "Test" (don't set the slug or description)
2) create a tag named "Test" (don't set the slug or description)
3) create a group "term test"
4) edit the Terms
5) select "Test" under Tags taxonomy
6) click "OK"
7) click "Save"
8) go back to Custom Fields page
The Taxonomies section for group "term test" should read: "tag: Test"
The Taxonomies section for group "term test" reads: "category: Test"
My guess is that it is using the term_id instead of the term_taxonomy_id. A way to fix this without necessarily breaking backward compatibility would be to record the term_id as well as the correct taxonomy.