jQuery can reference objects in the HTML document by means of ID or CLASS attributes. When you have a document object -- like in this case an anchor tag
<a ... > -- that has the class attribute value set to "fancybox iframe" without the hyphen (which will be seen as two different class names) it can be targeted by jQuery with either
jQuery('.fancybox.iframe') where that dot before the value means jQuery should be looking for class names.
But when you used the first code I gave you, with
jQuery('.fancybox-iframe'), the script could not find any instances of objects with class="fancybox-iframe" (which is seen as one class name) because there were none.
On my test site, where I use class="fancybox-iframe" for all iframe links (you would need to activate the iFrame option on your Settings > Media page if you want to use fancybox-iframe on your site too) the effect should be visible too. Try the second and third link on http://demo.4visions.nl/easy-fancybox-sandbox/pdf-embeds/ for instance...