moogyb

It was just an odd coincidence and nothing to do with WP Super Cache.
I solved it by getting my host to restore to the backup from the day before.
What happened was that I used Yoast before updating their plugin and some weird javascript thing appeared.
It was discovered a few days ago that they had a vulnerability to cross site scripting. All fixed now!
I have now installed Wordfence (I have no idea how I didn’t know about this before) and deleted anything old like the themes you are given when you start.
From what I can tell the best way of staying secure is by making sure everything is up to date.