I have an email server which is exposed (that is without any firewall) to the Internet. However, I have patched the Windows to the latest and didn't install any other software other than the email server software.
Is it safe?
I would say, probably not.
You need a firewall of some sort, although a router based firewall would just about do, and even the inbuilt windows firewall is better than nothing (although I still recommend something else instead).
and you REALLY need AV software, especially one with the capability to scan emails received by the mail server.