Advanced Free Shipping only works to make the whole order free (when certain conditions are met).
Conditional Shipping and Payments looks like it will do the trick, at least the description describes what i want.
I was really hoping there was a way to do this in the basic setup, but it kinda seems like there is not.
Thanks for your reply and suggestion.