I just noticed this same issue today myself.
I'm running version 3.4.2 with three plugins (WP Super Cache, WP No Category Base and Widon't), none of which should alter the post editing page. The same thing happens whether I'm using my own custom theme or I switch it over to Twenty Eleven.
The custom field info does appear to be saved. If I save a post after adding a custom field, that custom field info shows up when I return to the post editor.
Seems to me that it's probably just a problem with the AJAX interface controls.
First, I enter some custom field info.
I click "Add Custom Field" and all that seems to happen is the "Add Custom Field" button disappears.
That info does appear to have been saved properly, though, because when I click "Save Draft", everything appears as I'd expected it to upon clicking "Add Custom Field" in the first place.
Okay... So I now go on to add another custom field info.
When I click "Add Custom Field", the "Add Custom Field" submit button again disappears, but this time it reappears (slightly restyled) above the "Add New Custom Field" form. Exactly where I'd expect the custom field info I just added to appear.
And, once again, if I save the post, the page refreshes to show that everything seems to have saved properly.
For the time being I guess saving the post after each custom field is entered will be my workaround.
But this seems like a legit bug.