I did a fresh install with the standard theme to see if I got the same problem. I inserted a string “Foo & foo” into $formatted_data in the hook cm_typesense_data_before_entry and it rendered properly.
I did some more investigation and found out that when I’m using get_the_terms() or get_the_title() the returning string is encoded.
Because of that, I tried to run the wp_specialchars_decode() function on my resulting values, like: