Sorry; got it working. It was a plugin causing the errors — the Redirection plugin, in case anyone was wondering.
By the way, the server is on BlueHost if that makes a difference to anyone…
Yep. Same issue with the twenty ten theme.
Did it: http://pastebin.com/PM4UXW6p
I tried the theme out on a different WP installation, another site, on another server and it works fine. Does that mean this is likely a host issue??