I did some bench marking on the Blogroll RSS widget and recorded load times for the widget area of over 5 seconds.
Here is an upgraded version which I tested with a 0.002 second load time.
The performance boost is gained by caching the widget content for a time frame of 3 minutes. It would preferable to process the caching via a cron job if possible, as whoever triggers the cache to refresh will get a very long load time. In the mean though, this should at least fix the problem for the majority of the people visiting the page.