After upgrading from version 3.1.9 I had the following problems on my 2 blogs.
PHP 5.2.11/MySQL 5.0.81 (No problem with time outs nor memory exhaustion.)
Time/Memory used between version 3.1.9 and 3.2:
Test blog (running WP 2.9 beta 1) (6 posts)
3.1.9 = 2.01 secs/20 MB 3.2 = 12.5 secs/26 MB
4 year old personal blog (running 2.8.6) (approx. 1000 posts):
3.1.9 = 5.03/21 MB 3.2 = 12.5 secs/36 MB
I had updated the plugin automatically. Reactivation took around 20+ sec on each site (auto-update reactivation for previous versions was around 5 to 6 secs).
Attempting to access the XML Sitemaps options page redirected me to a page stating that I didn't have sufficient permissions to access that page (I'm the administrator so I found this hard to believe). After reloading 3 or 4 times the options page finally came up. Same behavior on both sites.
Non-cached home page rendering on both sites took approx 8 sec with version 3.2 activated versus a normal .6 to .8 sec (tested with Firefox 3.6 RC, 3.5.5 and IE 8.0). These times refer to dynamic generation of page not total browser rendering time.
Downgrading to version 3.1.9 returned both sites and plugin behavior to normal.
Note: I haven't had any problems with the previous versions and unfortunately, nothing is showing up in the error logs so I'm unable to provide any more information other than the above. If there's any other test I can perform please let me know.