First, the error on your website is related to the fact that the word "feed:" has been added to the actual RSS feed URL. This is a future standard that will work at some point time, just like "mailto:" works right now. To fix this, just edit your footer.php or sidebar.php or whatever your RSS feeds appear and remove the "feed:" portion of the URL.
RSS is the mechanism by which you syndicate content on your website to the world. You can syndicate an excerpt or the entire content. You have that granular control.
People can write websites that "suck" the RSS feed from various other websites and show their contents. Bloglines, for example, is a web based reader does exactly this.
When people talk about a "reader", they are talking about programs like Sharpreader, Firefox and others. Check this list to see other readers for other platforms.
What you would do is download one of these programs and then enter the FEED url, in WordPress, it could be http://<yoursite.com>/feed/rss2 into the program. It will then go and take that XML output you see in your browser, intrepret it and show you the website's content.
This way you can take a single program and put in the feeds of all the websites you usually frequent, Slashdot, Yahoo News, Gizmodo, WordPress Dev blog, your own blog, your friend's blog and so on.
This will allow you to keep up to date with all the contents on these websites without having to actually visit them with your browser.
All WordPress blogs have RSS capability by default, you do not have to do anything. If your blog is of interest to readers, they might just subscribe to your feed to keep up with it rather than visiting your site.
I hope I've made things a little clearer.