Thanks for the reply.
I tried the fix they did not work.
Then I tested sendmail using a very simple form mail code. It just worked fine, even the sender is not from my domain name. So I think maybe it is not my hosting refuse to send email but something wrong with WordPress itself?