If this is 2.1, I'm not sure what I know how to do in 2.0 will work.
That caveat entered, here's what you do to enable this sort of functionality:
Before anything else (even the doctype!) on the page where you want the wp functionality to appear, place this code:
Note that that server-path string is the sort one has on some servers; there are others. If you don't know what your specific server-path really is, ask your host support staff.
Once that's in place, then put the "bits" where you want them in your page.
Couple other caveats:
1. I haven't ever tried this with widgets. It may work fine, but I can't say definitively.
2. If the wp install you want to scab the widgets from lives on another domain in your server space (or sometimes even in another area of your folder structure) you may run into the "open_basedir restriction". If so, the only thing I can tell you to do is use wpget - a specifically tweaked wpget - and if this happens to you, post back and we'll go from there.