Solved. Got back in through the .htacces file and noticed the login url had been changed (not by me). There was a /? in front of the custom url field for me to type in and that was part of the custom url I used before things went weird. The weirdness seems to come from that ? vanishing from my url.
No caching plugin installed yet btw.
Like the plugin very much, but have the same or a similar problem here. Used rename login page and that was working. Added maintenance mode, things were still working. Added forced log-out, still okay. Started working on the content of the site, got logged out a few times and was able to log back in again just fine, but all of a sudden I am now stuck with the maintenance mode page instead of a login screen. I did not change any settings between login attempts.