I too have run into the dreaded slow loading admin panel.
It takes forever(~1 min) loading up. The worst is when I update a post, all the little .gif's come on one at a time, and they're in no hurry!
Using to timeline tool in Chrome, I saw that color-fresh.css and l10n.js took around 30 seconds to load. I haven't seen any other post discussing these particular files.
I have disabled all my plugins of course. I'm running suffusion theme, but so are my other sites which have no problems.
I really hope, that the details about those two files might bring some details to light.
Thank you all in advance!