Further investigation into this problem has shown up fopen with URL wrappers to be causing the problem.
This only seems to occur with PHP 4 on my hosting package - if I switch to PHP 5 it all seems to work. With PHP 4 - the only transport I have available is fopen, which means that the system will bomb if I allow it to use this...
I can now use http.php, after commenting out the option for checking WP_Http_Fopen::test() in the file - around line 103 in the _getTransport function - I could hack the test a bit rather than this function, but you get the picture: If a transport is messing you around - comment it out, or comment them all out then uncomment 1 by 1 until you find the culprit.
Now I can still use http methods and don't have to "return" at the top of the request function. (PHP 5)
The WP_Http_Fopen::test() only tests for allow_url_fopen - which is valid, but on my hosting, there is no way of testing if a URL is blocked by the firewall. Not sure about the slowness of Curl, but this works fine. One thing I have noticed after adding debug into the request code is that the dashboard page makes 4 requests to the same URL so if 1 call is slow, then the others maybe slow too. I know that the dashboard is a modular thing, but hey couldn't WP cache the results of these requests or something so that the returned data could be reused...
1. I would recommend allowing users to select which transports to use in order to override the system - it's alright checking, but because of the issues with external library versions, etc. It might be nice to allow people to tweak the settings to suit them.
2. If you are not caching the URL requests offsite, please do, or make it a lot easier to turn them off...
Thanks
skila