Hi all. The more I think about this, the more I see it as a bug in the software.
If a writer uses quote marks or apostrophes in a title, then the permalink URL that's created actually includes quotes or apostrophes. That's really bad URL formatting and although they seem to work, causes problems if you want to share posts. Plus it looks really ugly.
I'm curious as to why WP allows this at all, and whether I'm just one lone guy freaking out about it. Seems like common sense that the "pretty url" feature should strip anything other than alphanumeric characters.