I found a solution – a friend helped me.
Anyone who know anything about this?
Ensure your directories are 755, and most files – except for explicitly required otherwise are 644
As extra info – I did set it up as an ErrorDocument in .htaccess, and it does show when I write some random URL. However when visiting /wp-admin, which I have set to go to a subdomain with document root set to the same directory as the wp directory, it’ll just go to “Item not found”.