I think post-formats can take much of the more straightforward functionality that one would traditionally obtain with either post categories or custom post types, with much less hassle. Normal editors may well forget to select the right category everytime, when that's important for functionality and not just categorization; and custom post types may for many tasks be a bit overblown and more work for the developer.
In a recent site I decided to use asides as just that, asides, to appear only in a widget in the sidebar. But then I had to exclude them from all queries except for its taxonomy and the widget query (through pre_get_posts), from next and previous links in single posts, and from link rel=next and link=prev links in the head. For the latter two tasks, I need all asides to belong to a given category to be able to use the exclude_categories arguments, and I don't trust my users to always remember to assign it.
My suggestion is twofold:
- replace category arguments in most or all functions by tax_query (at most letting legacy category arguments as shorthands and for backward compatibility)
- provide much more support (including visual support) in the core for editing various post-formats, along the lines of the CF Post Format plugin.
As I see it, post-formats are useful to separate various functionalities, not just for having different presentation styles. They are like predefined custom post types.