I usually go to host and remove the rss file there.
I then go to the header and delete the folowing lines.
<link rel="alternate" type="application/rss+xml" title="Feed Title" href="Feed Location" />
The other link is to atom, usually one on one line the the atom one below that.
If there are Icons in the theme index I remove those as well.
Thats what i do but save your code in notepad you take out so if all fail (it should'nt) then you can put it back just leave a big space where they will be missing till you save and preview. if all go ok then close the gap.
DO NOT TAKE OUT ANYTHING ELSE ONLY THOSE TWO LINES.