It's apparently dependent on the file owner and/or permissions. After making sure I'm the owner of all files in the /wordpress tree the upgrade process no longer asks for ftp connection information and upgrades both wordpress and auto upgrade plugins correctly.
To make sure the webserver is the owner I use "chown -R www-data:www-data /var/www" on my Debian home server. For Hosted sites it's unlikely it's an owner issue. I use a local network to manipulate files so sometimes they end up being created and owned by root.
From FTP it may be that you need to change all files to 0777 rwxrwxrwx (Owner, Group and Public Read, Write, Execute) using chmod or your favorite FTP program. Then try the upgrade again.
Once the upgrade is finished it's a good idea to set permissions back to default, which should be sufficient to do the upgrade correctly. I believe the WordPress default permission is 644 (rw-r--r--) which means owner can read, write and execute, and group and public can only read. For security purposes it's a good idea to at least be sure you wp-settings.php is set this way in case php barfs and spews your database login information all over the net. ;)