Error: 302 Found

I’ve just installed the latest version of WP. Now, whatever I do I get this 302 page with the message:

“Found
The document has moved here.

Additionally, a 500 Internal Server Error error was encountered while trying to use an ErrorDocument to handle the request. “

The link behind the word “move” goes to the same URL as I am trying to access and if I click it (and thereby reload the page) it all of a sudden works. This is extremely annoying so what do I do about it?

Edit: I’m editing this post as now I can’t even log in and all I get is the 302 page that won’t show any other contents than the error message.