If you need to create an additional custom table, you can certainly just do that. This page, intended for plugin developers, will show you how to create custom tables in WP.
However, you really shouldn't need to do this for multiple sites. Are you already taking advantage of multisite?
If you just need multiple blogs/sites to work together, use multisite: http://codex.wordpress.org/Create_A_Network
Multisite will do all the dirty business for you - no reason to go rummaging around the database trying to reinvent the wheel.
If you are already have multisite, and are actually need multiple multisites, or if multisite for whatever reason doesn't seem to fit your needs (can't imagine why), and you absolutely need multiple instances of WordPress, you probably really should to have multiple databases.
--Prefixes ARE the answer, weather you like it or not.
That said.. there is a way to have multiple instances on a single database. I'm sorry if you dont like the answer, but it absolutely has to do with the table prefix.
I have not done this but I presume if you just install them separately as you normally would, and then just point both of them to the same database, you could just provide each install with a different prefix - can't see why such a thing wouldn't work (its essentially what multisite does)
FYI: The reason its not simple to modify the schema is because your really just not supposed to. Developers in general should try to limit their direct handling of the database, because it can change over time from version to version while keeping the API more consistent. Not only that but plugins and WordPress itself expects the options table to be named a certain way, and changing it would break WordPress let alone plugins. Any new custom options table you might create would be invisible to WordPress, and not integrated into its API.