I installed WordPress and was having SERIOUS performance issues, particularly in the admin screens where delays of upto 2 minutes were being experienced. Well, three hours later I think I've stumbled across the cause of the problem. I had noticed that every now and again the URL in Safari would include the port number 16080, which puzzled me.
This port is something to do with the performance cache option for Apache, I turned this *OFF* and everything was blazing fast. Very very odd.
I can't pretend to know why this worked, but if you're having problems with WordPress performance on OS X Server then give it a try.
The option is in "Options" tab when editing a site using Server Admin.