Just wanted to post this in response to this topic: http://wordpress.org/support/topic/84459?replies=3
I had the same problem recently where an extra apostrophe appears all over my site (keep in mind, there is only one apostrophe in the data in the db), and if I made an edit, the extra apostrophe would be inserted into the database, thus showing up as 4 apostrophes on the site, etc.
I had my host look into the problem, and they solved it. Basically it has something to do with the Magic Quotes in the PHP settings.
They remedied this by inserting these two lines in the .htaccess file:
php_flag magic_quotes_gpc Off
php_flag magic_quotes_runtime Off
Hope this helps anyone else having the problem (I couldn't find the solution myself!)