So I created a basic achievement. Once it is trigger, the box shows up. I had an issue where I couldn't close the box but followed Paul's steps of copying over the feedback-achievement-unlocked file into my theme folder structure. The box now closes.
But, after closing the box, anytime I change pages, the same achievement box appears again.
Any input on how to show achievement box only once. I'm sure this is the intention of the plugin.