Outputting the link with regards to the custom field is the main thing.
I have no experience with the plugin you mentioned, so I cannot be much help there.
But, you plop your mp3 (or whatever) URL into a custom field ... but now what? By default that doesn't get displayed anywhere - so you would need to output the custom field somewhere to be useable by your reader.
And then moving on to your RSS feed - a custom field wouldn't be spit out in the feed - you need a way to get it out there. YOu could probably call the custom field meta data, and display it in your feed using a function.
Basically - putting the URL into a custom field is only half the battle - you need to manipulate that data to be used.
Incidentally, this would probably be why many people use a podcasting plugin - I use mine so I just put in a URL and be done. I get an audio player in my post, I get a podcast only RSS feed, I get auto pings to iTunes, etc. Admittedly, these plugin are often way more than many people need, but they do offer benefits!