After almost 12 hours, was able to finally fix the WordPress 2.5.1. "still needs to upgrade to 2.5.1." issue. Most of the suggestions were mentioned already but here are links and steps that might help.
MAKE SURE YOU BACK UP YOUR DATABASE BEFORE DOING THIS!
After upgrading my WordPress version 2.5 to 2.5.1., the dashboard is still telling me that I'm still on version 2.5. That's what brought me to the forums and eventually I found out I have those _new, _old, .pngg.php, .jpgg.ph, etc files in my content directory. So I immediately deleted that. For some reason though I can't find the 'wp-info.txt' file, even while viewing the hidden files (using filezilla).
This was the most helpful link I got:
1. http://wordpressphilippines.org/blog/has-your-wordpress-been-hacked-recently/ - read carefully and follow as instructed.
2. Make sure you delete the phantom "WordPress" user. To instantly check if you have that user, go to your WordPress admin "users" page, enter "WordPress" under Search Users, if you DO NOT get a "No matching users were found!" you definitely have the phantom user in your database. You will notice a weird blank box appearing if you have that user. Another easy way is to Write Post, scroll down to Post Author, if there's an "invisible" author, that's the phantom WordPress user.
Check the link provided on #1, and delete it by accessing your database.
3. As mentioned in one of the entries above, look under database wp_options the VALUES 'active_plugins' and 'deactivated_plugins'. It is very likely you will notice a plug-in that's not supposed to be there. Here's what I get in mine:
i:0;s:117:"../../../../../../../../../../../../../../../../../../../../../../tmp/tmpw6d0cG/sess_dea436 and so on (a very long plug-in entry)
Remove that entry. If it works, then fine. If it doesn't, you can delete the entire line entry (active_plugins and deactivated_plugins) and WordPress will automatically create new entries BUT MAKE SURE YOU BACKUP FIRST! THIS HAS NOT BEEN VERIFIED TO WORK FOR EVERYONE, but that's what I did on mine and it worked. When I enter the admin page, I just have to re-activate the plug-ins.
Check your dashboard. You'll read the message: "This is WordPress version 2.5.1.". BACKUP your database immediately to help you against new or missed exploits.
What I noticed is that removing the offensive files will still give you the false version message in your dashboard. What cleared it are steps #2 and #3.
I strongly suggest you download the WP Security Scan plug-in (newly updated, just google it). It checks your writable directories as an added insurance that your directories have the correct chmod. If you tend to edit your themes often, don't forget to give them back their original permissions for added security.
I hope this helps. BACKUP first before trying.