I erased it by going into FTP
I also erased the index.html files in the supercache folders.
I had to give the folder 777 permission and then deleted it.
MY INITIAL POST BELOW
I have disabled and even deleted the plugin but it still active! It is showing a white page and the title which I entered into input field in plugin setting page.
I have used this plugin on older versions of WordPress and it worked fine. Now I just want to get rid of it(Until bug fixed)!
(UPDATE:Not plugin creators fault that plugin stayed active after deleting, it was the Supercache plugin which is activated automatically with WordPress 3.5)
Do I have to erase any trace of it from my database?
[UPDATE: All traces erased by deleting it from wp admin>plugin page.]
What happened :
It was working in preview mode but when I checked browsers where I was not logged in, (Google Chrome and Safari) it only showed a white screen and the Headline text.
Is it a 3.5 issue?