I ended up leaving 1and1 over this issue.
Support's official answer is that each plan level has a fixed memory limit and to check the standard settings despite the fact that 1and1 mostly ignores them. My plan's limit was 80MB, however support did not even acknowledge that there could be an issue when an account is bombing at 1/10th its limit.
To completely take WP out of the equation I made a two line PHP test script for testing. The script assigns a bunch of characters to a variable and then reports the scripts's memory usage. I adjusted the amount of characters in the variable to increase memory until it failed.
The account with trouble was failing at 8MB. I ran the script several times through out the day and it went as high as 11MB, but then went down to 8MB. I have a second 1and1 account with the same hosting plan. I ran the script on the second account and it worked fine up to 28MB.
I ran the same script via the shell using the php5 binary and was able to successfully run up to my account's 80MB limit. If I then ran through the browser, it bombed at 8MB.
This allowed me to confirm:
1. This has nothing to do with WP. If you are having this issue, don't go breaking a working WP install unless you changed or added something that would increase your memory demands.
2. The limit is different between accounts of the same plan and also different during different runs in the same account. Best guess is that they are doing this for resource mgmt reasons. The real problem is that they tell you exactly the opposite and claim it is a problem with your software.
3. The limit is coming from the web server module and not the core PHP install. The php command line binary in the shell does properly follow the standard php settings.
There are many, many stories around the internet about 1and1 customers chasing phantom php memory issues that they just cannot pin down. I believe that this dynamic memory limit issue is the cause.
After spending the better part of a day testing and troubleshooting, I again contacted 1and1 support to tell them what I found and to ask if their memory limits were dynamic.
Their answer to my test results and site being down for days? ... an FAQ about how to uninstall WP plugins.
The complete and utter lack of support is the reason why I decide to leave 1and1. They did not even bother to look at my account, read my emails, or put forth any effort to resolve the problem. I hope that no non-tech business users that rely on the service for their income hits this problem.
I setup a Dreamhost account, copied the files and DB, and was running at 100% in under 30 minutes.