Some of those "widgets" are not really widgets. If you examine your theme files, you will likely see one named sidebar.php -- check and see which ones are written into the code and won't let you modify them.
You can go into the design settings all day long and they're never going to go away that way if they're written into your template, and reverting back to 2.3.3 will certainly never work.
The code which generates those things in your sidebar is likely there, tho I caution you on commenting on php files unless you know what you're doing, or you run the risk of creating php errors.
What do you want? A blank page with no sidebar at all? You should create a template for a blank page and add it to your theme, then republish whatever page you're trying to fix. Unless you really don't want a sidebar, and in that case I don't know what to suggest.