When I add and remove widgets, only some of these changes are showing up on the site. It seems like a caching issue, but that doesn't explain why some of the changes show up and some don't. I'm testing with simple text widgets and some appear after I create them and some don't. Also, when I remove them, some stay on the site. This is a huge problem because I have a badly-behaving widget that I've removed in every way possible (completely deleted, even from the inactive widget area), but it is still on my site, breaking my columns. I've deleted the browser cache and deactivated my caching plug-in. The widget is still there on other computers too.
What is funky about widgets? What could make my adds/removes effective just some of the time?