This is a styling issue with your theme styles causing the problem. In my FAQs I mention that the theme styles play a big role in the look and layout of the product - and this is one of those times.
To fix it, you need to add your own styles to override the theme. Go to the settings for the plugin and at the bottom of the page, check the 'use my styles' box and then add this to the styles below it:
border:0 none !important;
That should fix it for you.
If you have any additional layout issues, you will need to get someone with CSS experience to help you style the product appropriately (as we do not generally help with layout/styling).
Also, I don't recommend adding the button image the way you did, as it will be overwritten when the plugin updates. We hope to have a way to add your own buttons in the near future... but remember - until then, you will have to fix your button every time the plugin updates if your put it in the plugin images folder like that.
Best of luck and warm regards,