Well, actually you need to refresh the page to get the Deactivate button once you activated one avatar.
I've worked on an upgrade of the plugin that 'fixes' it, just pushed a beta on github.. If you can test it, and tell me if nothing broke, then i'll upgrade the plugin in WordPress repo
here's the change i made :
here's a zip to the plugin, make sure to keep the title of the main plugin folder to