I agree with Odolyte, something needs to be done in that department, even "the" Matt knows that. They had also a "job opening" in the Google Summer Code competition, for that, but unfortunately nobody accepted the offer.
What I envision is something similar to what Linux or Unix have: some sort of access control list where I can say "this user can post in this category, read this post but not edit it, etc" Each 'object' in WordPress (posts, categories, plugins, etc) should be associated to its 'umask' (something like rwxrw-rw-), have an owner and a group, just like in Unix or Linux, not to mention modern flavors of Windows and MacOS.
This would make WordPress a REAL content management system. Think about it: even with those plugins RVoodoo mentioned, you can't reach that granularity.
Here one of the scenarios I have in mind: a big organization's website. Each department of the organization should be the owner of a section (category? page?) of the website, and be allowed to edit articles only in that category. Maybe inside a department they want to implement a workflow process, where user A can submit new articles but not edit published ones, while user B can published pending articles, edit published articles.
I know, you can already do something 'similar' to that with WordPress, but it's just... SIMILAR, not exactly the same thing.
I hope WordPress 4.0 will include some of this granularity in the core :-) Enough with the Media Manager, already... JUST KIDDING!