The issue is due to the PHP version we require for the plugin, which is 5.3.0. Anything below that will error out. Usually if you ask your host to upgrade your PHP version to 5.3.0, they will do so without an issue and with ease and no interruption to your site. If they cannot or you do not want to upgrade, you can always go to the "Developers" tab on the plugin page (far right tab), and download version 1.0 that is compatible with older version of PHP.
I will be releasing a new version of the plugin today that explicitly states this upon activation and within the README to prevent issues like this from continuing.
Usually upon activation if the plugin produces an error, such as this, WordPress will deactivate the plugin for you automatically. Not sure exactly why it didn't in this case, but an easy fix would be to just remove the plugin and WP should automatically deactivate it with the files totally gone. Then from there, you can either get upgraded to >= 5.3.0 or download version 1.0 to work with your older PHP version.