FancyBox taking several seconds to open

Is there anything I can do to speed up the time it takes fancybox to open?
granted I’m linking to full pages, but it sometimes takes 3 seconds before the user see’s the box appear.
The site I’m currently testing it with can be found here: http://chiseledimages.com/misc/zangla/pets/
I have the left sidebar links all as fancybox.

If it’s just unreasonable to expect better performance then just let me know.
thanks.

http://wordpress.org/extend/plugins/easy-fancybox/