I updated all my sites to the latest version of the SEO plugin and found that all the breadcrumbs outputted the raw html.
The SEO plugin used to allow html in the breadcrumb settings:
Separator between breadcrumbs:
Anchor text for the Homepage:
<i class="icon-home"></i> Home
This allowed me to get my theme (based on Twitter Bootstrap) working the way I wanted (i.e. get breadcrumbs to output as a list and using a home icon button)
I noticed previously that the breadcrumbs settings page didn't like HTML and cut it off at the ", but it still worked really well on the front end.
Now it seems that the breadcrumbs converts all the entities directly and so I can't use HTML.