For a project I have added a lot of categories, a few thousand. In the front end that doesn't seem to be a problem, but clicking some categories in the back end is terribly slow. It seems as if each choice is processed at the spot, not after actually saving the post. When I click on the checkbox of a category I have to wait for a second or two/three to see the box being checked. When I want to check a dozen categories... that isn't really going smoothly.
I think this has to be altered in the core, but I'm no php programmer ;)