I found the problem! (Actually the hosting provider found it for me) – I copied my old cgi-bin folder from the different hosting provider and overwrote the existing one. Then, frustrated I just deleted it all (including the cgi-bin folder).
That was the problem. The hosting company copied it back, it works now.
(Thanks for the encouragement with the film 😉