Problem with plugin after upgrading to 3.4

I wanted to put links to affiliate sites on a rotating gallery so got the plugin which – I know! – said it had not been tested for the upgraded version of WordPress. Well I activated it and then deactivated after I saw it really screwed up my header and pages listed there. Now I get this message when I try to log in:

Warning: Cannot modify header information – headers already sent by (output started at /home/shahanj/public_html/wp-content/plugins/money-maker/wordpressPlugin.php:109) in /home/shahanj/public_html/wp-includes/pluggable.php on line 881

I don’t know how to get around that to get into my website to delete the plugin, post or anything! Any ideas?