It's a bug that has been mentioned in another thread:
The redirects appear to be deleted on the admin screen, but are still present in the database. If your host allows you access to phpMyAdmin, you can go in and delete them there. If not, speak with your host's support and ask them to do it for you.
Unfortunately, the other thread was marked as resolved. Even if this solution works for you, please don't mark this thread that way. Should leave it open so the developers get a chance to see and correct it.
Hope this helps! :)