Blog: How to Setup Your Own Mail Server that Will Deliver

There are many reasons why you might want to have your own SMTP server. But there are also many reasons why lots of businesses outsource email sending to third party services. One of the big problems with having your own SMTP server is that it is quite hard to set the whole thing up to get a solid delivery rate. What does it mean? It means that if you just naively install an SMTP server and try to send emails through it, many of your emails will not be delivered. Moreover, you will not even know that your emails did not reach their recipients. If you have decided that you want your own SMTP server, this article will help you reach a solid delivery rate – i.e. your emails will be delivered to existing mail boxes and will not get deleted as spam. If something is not configured perfectly, your emails can be lost completely without a notice, moved to Junk folders, or not accepted for a delivery. So, if you are asking following questions: How to deliver an email? How to setup MX server? How to increase the delivery rate? Just keep reading.

Note that the methods described in this article will work for you only if you are a legitimate business and do not intend to send spam. If you do send spam, it is a good thing that your emails are not delivered, and even if you set everything properly, many of your mails will get deleted.

This article only discusses sending emails. Receiving emails with your own mail server is another wide topic. We will use the term mail server universally for all roles and names such as mail exchanger, MX host, MTA (mail transfer agent), or mail relay.

Choose the Right Software – DKIM Support is Required

You would think that every professional mail server software will simply support all features that are needed to deliver emails today. While this is true for most of the mail server software out there, this is not the case of Microsoft IIS SMTP Server. This SMTP server by itself does not include a crucial feature called DKIM (we will discuss it in a greater detail later in this article) and you can search the Internet to find many third party products that will add this feature to your IIS SMTP Server, but these are not cheap. If you are on Windows OS and unless you want to pay for having DKIM feature in your server, you want to avoid IIS SMTP Server. You can try hMailServer for example. It is free, open source, mail server and it works well, DKIM included.

Recommendation: Whatever mail server software you choose, make sure it does support DKIM.

Connection to the Internet – You Need Dedicated IP Address and PTR Record

In order to fight spam, people developed number of ways and heuristics to distinguish between trustworthy mail servers and servers used for sending spam. To have a long term excellent delivery rate and thus being able to deliver emails today, means to comply with most of these techniques. One of the feature that good mail servers do have is that they are sending emails from IP addresses with a valid reverse DNS record (PTR record).

If your mail server wants to send an email, it finds a target mail server, which is responsible to deliver messages to the given recipient’s address, and attempts to send the email through it. If the target mail server sees that the request comes from IP address without a valid PTR record, it can refuse to accept any messages from you. Many mail servers also require that the domain name in PTR record corresponds with the domain name part of the source email address of the message to be sent. For example, if you are about to send an email from xyz@example.com, you might need to have your PTR record set to something.example.com.

Moreover, some email systems go even further than that. They do not accept any email if the PTR record does not contain or start with mail, smtp, or mx. Although this limitation is not very common, you can only increase your chance to deliver your emails if your PTR record complies with it. This is sometimes called reverse hostname naming convention or rDNS naming convention. In our example with an email from xyz@example.com, you would need to set your PTR record to mail.example.com to increase your chance of delivery. In case of two mail servers, you can go with mail1.example.com and mail2.example.com and it will work just fine.

In order to be able to set a PTR record for your IP address, it is usually required that you have a dedicated IP address for your server provided by your hosting or ISP. It is highly recommended that a single IP address has at most one PTR record and this is why a shared IP address is not a good idea.

Recommendation: Make sure your server’s IP address has a valid PTR record, and that its domain name corresponds with the source email address from which you send emails. If you want to further increase your delivery rate, you may want to set your PTR record’s FQDN to start with mail, smtp, or mx.

Do Have Sender Policy Framework (SPF) Configured

Another action that the target mail server can do to verify your trustworthiness is to check your SPF record. SPF record is a TXT record in DNS that defines which hosts are allowed to send emails with source address set to a particular domain.

A SPF record is a TXT DNS record that starts with v=spf1. Then there is a list of parameters separated by spaces. The full list of parameters is described in SPF Record Syntax. There are also applications, such as SPF Wizard that will help you create your SPF record correctly.

Whatever way you construct your SPF, make sure that it is restrictive. This means that it defines a limited number of servers that are allowed to send emails on behalf of your domain, and forbids all other servers to do so. This is done by having -all as a part of your SPF record (usually at its end). What happens if you do not restrict your SPF record with -all? A minority of mail servers mail not accept your emails at all, but more likely scenario is that while counting a total spam score of your email, it will simply score worse. Another reason for having this restriction is that if someone attempts to send email pretending to be from you, it will not be delivered. If the restriction is not implemented, the attacker is more likely to succeed and have their fake mail delivered.

Examples

Let’s assume again that your domain name is example.com, and that you want to send email from address something@example.com. Let’s further assume that IP address of example.com is 198.51.100.123. Here are some examples of good SPF records:

v=spf1 a -all

This record allows your emails to be delivered by hosts which IP address is contained in A DNS records of example.com. Since we assume that its IP address is only 198.51.100.123, your mail server will have to run from this IP address.

v=spf1 a ip4:198.51.100.124 -all

This record allows your mail server to run from 198.51.100.123, or from 198.51.100.124. The .123 address is allowed by the a mechanism in the SPF record, while the .124 address is explicitly allowed by the ip4 mechanism.

v=spf1 a ip4:198.51.100.124 include:hotmail.com -all

In this example, we have extended the list of allowed servers even further. In this case, we allow Hotmail email service to send emails on our behalf too. The include mechanism requires the included domain to have its own valid SPF record. Hotmail complies with this requirement and thus hotmail.com email servers that can deliver emails for hotmail.com domain will also be allowed to deliver emails from example.com.

Recommendation: Configure your SPF record properly and restrictively.

DKIM and DomainKeys

You have probably heard about DKIM and DomainKeys. Good news is that you do not have to worry about DomainKeys. It has been replaced with DKIM entirely and not having DomainKeys is not a disadvantage today. Not having DKIM, however, is a very serious problem which will kill your delivery rate. So what is DKIM?

DKIM stands for DomainKeys Identified Mail, a protocol that introduces digital signatures to email delivery systems. It is just another mechanism that the target mail server can use to verify that the incoming message is being sent by someone who is authorized to do so. If the sender passes the DKIM check, it is almost certain that the source email address is not forged. The mechanism works as follows.

First, you need to generate a public and a private key that will be used to sign and verify messages from your mail server.
Then you configure your mail server software to sign all messages it sends out with the generated key.
You also need to create DKIM DNS record, in which you publish your public key.
When your server is sending an email to the target mail server, the target mail server sees that your email is signed using DKIM, it performs DNS lookup to obtain your public key and uses it to verify the signature in the email message. If it matches, the email is more likely to be delivered.

So, how do you generate the keys? There is very simple DKIM Wizard on port25.com. Just fill in your domain name and something that is called DomainKey Selector. The selector is just any simple string you want – e.g. dkim. The wizard generates you public and private keys. Other way to generate the keys is using OpenSSL’s genrsa command.

If you have your keys, you need to configure your mail server to use the private key to sign the messages. How this is done depends entirely on which mail server software you use. So check up its manual or try to ask Google. If your mail server software supports DKIM, it should be easy to find. It should be noted here that there are two so called canonicalization algorithms called simple and relaxed. You are likely to see them as options to choose from in the configuration of your mail server software. Possibly, you might be asked to decided which algorithm to use for email headers and which for email body. These algorithms refer to a way how your email messages are signed. You do want to use the relaxed algorithm for both headers and body. The simple algorithm allows almost no modifications to the email message on its way from the sender to the recipient. This sounds great except that in the email delivering world, the message is quite likely to be modified. Various headers can be added through various anti-spam and anti-virus mechanisms or proxies, minor reformatting of the body is also possible. This means that if you choose the simple algorithm, your email is less likely to be delivered.

Now you need to create the DKIM DNS record. It is a TXT record that is usually identified with v=DKIM1. Assuming your domain is example.com and your DomainKey Selector is dkim then the name of the TXT record must be dkim._domainkey.example.com. Its value can be something as simple as the following:

v=DKIM1; k=rsa; p=YOUR_PUBLIC_KEY_HERE;

The only required parameter here is p with your public key. What you need to make sure, however, is that your record’s value does not contain t=y as this would make your record say that you are only testing DKIM and that it does not need to be respected.

If you need to know more about DKIM, continue to dkim.org.

Recommendation: Configure your mail server to sign outgoing emails using DKIM. Use relaxed canonicalization algorithm.

Create postmaster Account or Alias

RFC 5321 prescribes that every domain must have postmaster mailbox and accept emails to it. There exist mail systems that do check this before accepting an email from you. So, make sure that you create a postmaster account or at least an alias on your mail server.

Use of no-reply Address

If you use a no-reply address, from which you send emails that recipients should not reply to, your messages might have trouble to reach some mailboxes. This is because some mail servers require the source email address to exist, otherwise they refuse to deliver the message. The email verification is done using an SMTP protocol to your incoming email servers (defined in MX DNS records).

There is a discussion on whether or not it is good for your business to use no-reply address. But if you choose to use this mechanism, then you want to create a mailbox for that no-reply email address, so that it appears to be a valid address if someone checks it.

Recommendation: If you use no-reply address, create a mailbox or an alias for it.

Do Not Be Open Relay or Open Proxy

Make sure your mail server software is not configured to be an open relay or an open proxy. This would mean that you let anonymous users to send emails from your mail server. Always require users to be authenticated before they can send email through your SMTP server. The Internet is scanned by robots all the time and you can be sure that if you open your server to anonymous senders, your email server will be misused soon after it is installed. You will end up on blacklists and will be forced to restrict the access to your server.

Test Your Settings

If you have everything ready, it is time for testing your configuration. Start with checking your DKIM record, simply using DKIM Record Checker. Just fill in your selector value and your domain and submit the form.

If everything is OK, you can try to send an email to a very handy verification tool by port25.com. In order to perform the check, you need to send an email either to check-auth@verifier.port25.com or to check-auth2@verifier.port25.com. The verification result is going to be sent to you to the address specified in MAIL FROM SMTP command, if you use check-auth@verifier.port25.com. If you use the second variant – check-auth2@verifier.port25.com – the result will be sent to the address specified in From header of the email message.

The most important part of the result email that you receive will look like this:

==========================================================
Summary of Results
==========================================================
SPF check:          pass
DomainKeys check:   neutral
DKIM check:         pass
Sender-ID check:    pass
SpamAssassin check: ham

If this is the case then you just passed the tests and your configuration is correct. Do not worry about neutral DomainKeys check result, we have mentioned already that DomainKeys is not used anymore, DKIM is neutral result means that you do not implement that mechanism. If you fail a test, there is a detailed information included in in the result email.

However, even if you pass port25.com verifier test, it does not mean that all mail systems will consider your DKIM/SPF records as valid. This may sound weird but Microsoft’s services such as Hotmail.com or Outlook.com are actually much more sensitive to the actual syntax of DNS record values. This is why it is a good idea to make extra tests – tests with real world mail servers. You should be able to pass test with Gmail easily. Simply create an account on Gmail and send a mail to it from your mail server. Then wait until the email appears in your Gmail account and view its source (how to do this is described in Google’s help center).

In the email headers, look for the Authentication-Results header, which should look like this:

Authentication-Results: mx.google.com;
       spf=pass (google.com: domain of something@example.com designates 198.51.100.123 as permitted sender) smtp.mail=something@example.com;
       dkim=pass header.i=@example.com

If you see there spf=pass and dkim=pass you passed the test. Other values means that something is wrong.

Now let’s do this once again, but with Hotmail account. There are many recipients that use Microsoft email services, so it is very important to make and pass this test. Send an email to your Hotmail account and view its source. Look for the Authentication-Results header. If you do not see pass values, your configuration is almost OK, just not in the format accepted by Microsoft. In that case, review values in your SPF and DKIM DNS records and make sure that you use only one space as a delimiter after semicolon, no tabs, no new lines, just one space. Also make sure there are no quotes in your record value.

Recommendation: Use port25.com verifier to check your SPF and DKIM settings, but also make a test with a real Hotmail account.

Avoid Blacklists

There exist many blacklists, commonly called DNSBL or RBL, that attempt to list servers known to send spam. Blacklists are used for automatic filtration by some mail servers, so once your IP address is on a blacklist, it may have very negative consequences to your ability to deliver your emails.

Even if you are not a spammer, it is possible that IP of your mail server appears on some blacklists. How is this possible? One of the common situations is that a network of legitimate business is infected with malware that sends spam from within their computers. One or two days of infection can easily lead to be blacklisted. Another scenario is that the static IP address, which your hosting has assigned to you, has previously been assigned to a client who sent spam. Your IP address thus could be blacklisted even before you install an operating system to your server.

Fortunately, it is easy today to know whether your IP address is blacklisted or not. You can use free mail server blacklist checker to perform a check against over a hundred of blacklists at once. If it is crucial for your business to be able to deliver an email, you might be interested in automated blacklist monitoring service that alerts you if an IP address of your mail server appears on a blacklist.

If you are not sure whether your message that you want to send out can cause you a problem or not, you might be interested in reading Controlling the Assault of Non-Solicited Pornography and Marketing Act – aka CAN-SPAM. This is United States law that regulates the sending of commercial email. Even if you are not a US business, you want to comply with this law because you can be blacklisted otherwise.

Recommendation: Check whether your IP address is blacklisted after your hosting assigns it to you, and then check it from time to time or have it monitored. Comply with CAN-SPAM.

Commercial Certification

Optionally, you can get yourself a certification from a commercial authority that will maximize your delivery rate. You will still need to implement the steps mentioned above, so you can not just buy it. If you care a lot about your delivery rate to Microsoft and Yahoo email services, you might consider buying Return Path Certification. It’s not cheap, so really consider if those emails you need to deliver to your customers using Microsoft and Yahoo services are worth a couple of thousands dollars per year. If so, go for it.

Note that without a certification, you will still be able to deliver emails to both Microsoft and Yahoo. However, some of your emails may be lost from time to time, or reach the recipient’s junk folder instead of their inbox; especially when you start sending emails from a new IP address that does not have a good long reputation yet.

Your Emails Will Be Delivered Now

That’s it! If you configured everything as recommended, you should now be able to deliver emails to most of the mail systems in the world.

Here is a quick summary of what is necessary to achieve an excellent delivery rate:

Choose mail server software that supports DKIM.
Run the mail server on a machine with dedicated IP address and set a valid PTR record for it that starts with “mail”, “smtp” or “mx”.
Create a restrictive SPF DNS record.
Generate DKIM keys, configure your mail server to use the private key to sign outgoing emails and publish the public key in DKIM DNS record.
Do have postmaster account or alias.
If you use no-reply address, create a mailbox or alias for it.
Make sure your server is not configured to be an open relay or a proxy.
Test your DKIM/SPF settings, do not forget Hotmail test.
Regularly check your mail server’s IP address against blacklists. Ask for removal if you are blacklisted. Comply with CAN-SPAM.
Optionally, consider buying Return Path Certification if you need high delivery rate to Microsoft and Yahoo emails.