Update: The problem has to do with how WordPress itself stores the Blog title in its database. It runs attribute_escape() on the value before it inserts it into the database. I think this is a mistake as from this point on the apostrophe is replaced with #039; in the database.
One solution is to manually edit the wp_options table's blogname entry and replace the #039; with the '. This seems to fix it and doesn't seem to break anything, however I haven't done extensive testing on it but it seems to work for me.
I have also ran into the same issue with another plugin, Subscribe to comments, that one requires the modification of the source code as it sotres the value correctly but runs it through attribute_escape() upon retrieval from the database.