Spent some time on this problem and here is how I solved it.
1. I used the PHP installer which does not contain all of the dlls and I was getting the error discussed.
2. I downloaded the windows zip file -> PHP 5.1.1 zip package ( http://us2.php.net/downloads.php )
3. I copied the /ext folder from the zip file into my c:\php folder (now those files are in c:\php\ext)
4. I copied all the DLLs from the root of the Zip file to my c:\php folder. The ext files have some dependencies on these files so I copied them all to be safe.
5. I changed the value in c:\windows\php.ini (this may be in a different location on your machine) for extensions to say -> extension_dir = "c:\php\ext\"
6. I removed the commments from the extensions section so the MySQL settings showed as:
The mysqli.dll is supposed to be used for MySQL 5.0+ i believe... so i made sure both were in there. I believe the original ini file didn't have php_mysqli.dll listed and I had to add it myself.
After these steps it worked for me. I did have to add index.php to my default documents list for IIS (and i assume you would do the same in apache). I also had some database creation errors related to default values not being set by scripts but for the most part WordPress is up and running.