Hi jmek, thanks for trying out NextGEN Gallery Optimizer!
Did you remember to select your stylesheet on the settings page, at all? By design the plugin strips NextGEN's sitewide scripts and styles and lets you insert your own where appropriate...currently a stylesheet (makes thumbs display in a line) and also the Fancybox lightbox effect.
Apologies if this was unclear, I'll make sure to set a default stylesheet in the next version and perhaps an admin message to that effect too.
Also, the plugin only supports the [nggallery id=x] shortcode presently, which came from code I developed for a client who had that specific requirement. However, I've been working on an update that adds support for ALL of NextGEN's shortcodes, which will be available as a donation-based upgrade through the next version's settings page for anyone who wants the extra features. I'm aiming to release it on Monday (19/03/2012)...so do keep an eye out.
Hope this helps.
Cheers,
Mark.