Remove all functionality from the core except for a basic framework that powers plugins, presentation, users, settings, basic functionality.
Most features such as posts, pages, categories, tags, comments, so on and so forth would be provided in "core plugins". The core plugins that represent core's current feature set would be offered as a bundle: you could download just the WP framework or the WP publishing platform. This way, core could be more readily deployed for any and all situations.
This would also allow core plugin developers to more effectively bring new data/content structures into the platform. The current plugin system would remain for plugins that don't modify or enhance the basic functionality of WP. "Posts" and "Comments" are core plugins, but "Footnotes" and "WYSIWYG Comments" are extension plugins.