After reading that an upgrade was available, I decided top just overwrite the 3 files mentioned for the security fixes (wp-includes/pluggable.php, wp-admin/includes/media.php and wp-admin/media.php).
After going back to my site I found it wasn't working and the blurb mentioned a problem with wp-includes/pluggable.php
I overwrote it again and the problem was still there so I decided to go ahead with a full upgrade.
After the upgrade I cleared my cache and went to my site, only to find it is even more messed up now.
There is too much on the page to paste it here, here is the site in question: http://shame.tuxfamily.org/repo/
I don't know what the problems is or how to fix it.
Of course I can just restore from a backup (and did) but then I am without the security fixes.
I've since tried the whole process 3 times and still have the same problem.