In my case I wrote scripts that completely backs up all themes, plugins and user created content but not anything related to the wordpress core. I then nuke all files in the domain directory, apply a full fresh installation and reapply the themes and plugins and user specific data.
There is possibly a file missing somewhere but then I would have imagined that I would have had problems on all the domains I upgraded. Fortunately for me the only issue I had was on the domains that showed signs of the exploit I referred to above.
I wouldn't have been able to fix these issues without shell or sftp access combined with access to the database user and wp-options table.
Obviously removing the user called 'wordpress' shouldn't fix anything so I believe the fix for me occurred when I manually removed the phantom active plugin from the database column for active plugins. This came in the form of a big string like ./../../../../../../../tmp. I don't know how the string got there, perhaps an installation bug or artifact or a plugin installed left it behind. Who knows. I didn not find this string in the databases of blogs that did not exhibit this upgrade issue.
When I removed that entry all of my plugins became disabled so perhaps I munged the edit. I re-enabled them through the admin console and it was fine as were the blogs in question. In retrospect I would have manually disabled all the plugins through the admin console and then checked to see if that string was still in the database. If so I would have removed it at that time. In any event it worked for me.
Whether or not all of this is attributable to a true exploit or an errant plug-in is still up for grabs IMHO. All I can tell you is that I found the http://wordpressphilippines.org/blog/has-your-wordpress-been-hacked-recently/
article, cleaned up per it's suggestions and all was fine. The active plugins thing was actually contained in a user comment on that site.
Two plugins that I use that have given me issues since 2.5 are the TinyMCE Advanced plugin that stopped working between 2.5 and 2.5.1 and the fluency admin console that worked fine for about a week then suddenly stopped working coincidentally around the date time stamped concurrent with the .pngg files. I have never been able to get fluency to work (firefox/css2) since then and have given up on it. The TinyMCE advanced was fixed with it's latest upgrade that came out a couple of days ago. These two plugins are common on all the domains hosted by me.
I hope this gives more insight.
I caution anyone mucking around in the database or file system to PLEASE TAKE THE TIME TO BACK UP YOUR WORK AHEAD OF TIME!! It should only take about 10 minutes and will be well worth the time and effort.