I haven't tracked this down to an interaction with another plugin yet but if I uncheck Enable on Pages or Enable on Home Page etc in Media Settings this stops WordPress from loading jquery.js.
I noticed the problem when one of my custom scripts stopped working on an archive page that I don't want light-box to load on. Rechecking solved the issue.
So, barring some other filtering the light-box script will have be loaded on pages where I don't need it.
Is it just me?