OK, it's NOT a WP bug, at least not that I can tell from my upgrade test.
I have a single VPS account and I have two identically configured websites on it. WP2.5 is on both. Both blogs are in subfolders. On the one that works, it is in a subfolder called /notes/. On the one that's broken, it is in a /wordpress/ subfolder. The .htaccess files are correctly configured for each.
I'm stumped. There's no reason why one should be different than the other. There are no plugins other than Akismet 1.4 active on either, these are fresh installs.
To be fair, there may be some hidden problem with the account that is borked. I previously tried installing WP 2.3.3 on that account and got array appearing all over the site. I've never been able to get WP working on that account in 5 minutes.
I've got better things to do with a Sunday afternoon. And yes, for each install I've nuked the database AND the db-user and started from a clean slate. Far from ideal, but so is a URL hack.
Anyone else have this experience and find a fix that doesn't involve a URL hack?