How to completetly delete the plugin?

Sounds like something is getting cached? Try disabling your WP caching plugins, or restarting your web server (to clear that cache if it caches), and go to the WordPress > Settings > Permalinks page and click save once (rebuilds the permalink map).

By the way, WordPress handles the slug name for feeds at a higher precedence than posts and pages. Because of this, you cannot have both a feed and a page with the same exact slug name. (imagine if you tried to create a page with slug name “atom”, “feed” or “rss2”, same problem will happen). We built special code in PowerPress to deal with this, you need to be in the advanced mode, then go to the “Settings” > “Basic Settings” tab, scroll to the bottom to the “Podcast Permalinks” option, select “Match feed name to Page/Category” then save your changes. PowerPress will then override the default WordPress behavior and allow the slug name to work for a page/category.