Mike, I noticed the securimage plugin you are using for the Captcha is still at v1.02 and they are at v3.0 now. I don't suppose you have considered updating it, if only for security reasons?
I ask because I am hoping the newer code is faster. I use your wonderful contact form (having donated) on a number of sites in a drop down slider on every page. Unfortunately, the captcha loads everytime and when looking at page load times, that single php function adds 332ms. I've disabled it for now, but hoping an upgrade is in the future.
Another option would be to load the Captcha via Ajax after they have a valid email, or filled in all the required fields.