I have now tested and confirmed this on three sites. Most of the sites I have developed no longer need this, as I now use Meta Boxes but some of the older ones I've developed are showing a problem.
When I try to add a custom field, everything works until I click Add Custom Field. Once it's clicked, the button disappears and an input is created with all of the other custom fields that just says "Add Custom Field".
If there are no custom fields at the time, then the button just disappears and nothing else appears. When I click Update/Publish/Save Draft, no custom fields are added.
Just a clarification: When I say "Update", I am referring to the button in the Publish meta box, and not the Update Custom Field button as that causes the same issue.
Has anyone else had this problem and is there a fix to the core code? At this time, I'm just going to revert the sites that rely on custom fields back to 3.4.1.