Or if you're hosting with a service that provided the install with an easy one-click, you might need to try your control panel for the hosting account. Look for web files, or directory or something along those lines where you can gain access to your WordPress install through the web browser.
Then you'll want to find the folder
wp-content Open that one and look for the folder named after that snazzy new theme you installed ;)
You don't have to delete the theme, but just change the name of the folder and WordPress "should" revert back to its default theme.
Hope that helps.