Activation Email does not send when asking for a new site from registration page

I opened a brand-new multisite WordPress with subdirectory form, not subdomain form.

And then I:

1. Logged in as super admin;
2. Enabled the option “Users can register a new account and create a new site.”, and saved;
3. And then I logged out;
4. And then I clicked “Register”, and I specified the new user name, and specified an Email address, and then I chose “I want a new site too” , and I clicked “Continue”
5. And then I specified a new subdirectory name, and I clicked “Finish”.

It prompts me that an activation Email will be sent to my Email address but it never comes.

But I had some technical background so I started to troubleshoot this problem.

1. I wrote a small script to test the PHP mail() function. It works.
2. I logged in as the superadmin and I manually added a new site and that action triggered a user creation. I received all emails thereafter. Four in total. 1) User creation; 2) User name and password; 3) Site creation with its URL; 4) Notification Email to super admin.
3. I logged out the superadmin and tried to register a user again. But this time I chose to create a user account only, not asking for a new site. This time I received the activation Email.
4. I inserted some PHP code into wp_new_user_notification() and ask it to log when invoked. I found that the function is not invoked if I create a user and ask for a new site. Because the log file was not updated. On the contrary if I create a user and do not ask for a new site the function is invoked and the log file got updated as expected.

Now the conclusion (and my question):

1. If admin creates a new user together with a new web site it will succeed, and all email sent properly, no need for activation and the user name and password combination will be sent directly;
2. If a user registers an account by himself/herself instead of by an admin, but not asks for a new site, he/she will receive the activation Email properly and he/she can create a new site thereafter without a problem.
3. If a user registers an account by himself/herself instead of by an admin, and meanwhile ask for a new site (by choose “I want a new site too” option in the registration page), he/she will never receive even the activation Email.

A WordPress bug, right? Please help me out by fixing this. This is completely nothing to do with Email server things so please don’t send that template, it won’t work if even wp_new_user_notification() has not been invoked.