Ok, thanks to carthik on #wordpress on IRC, I found the problem…
http://photostack.org/forum/viewtopic.php?pid=401
basically, change the line
$index = $_SERVER['SCRIPT_NAME'];
in the file:
wp-includes/template-functions-links.php
to:
$index = $_SERVER['PHP_SELF'];
There’s one other file with the variable $_SERVER['SCRIPT_NAME'] in it:
wp-settings.php
If I find anymore problems, I’ll change it there too.
I think this has a lot to do with running PHP as CGI.