papyri/md/smtp_relay.md

# SMTP Relay

## Contents

  - [Overview](#overview)
  - [Requirements](#requirements)
  - [Conventions Used](#conventions-used)
  - [SSL Certificates](#ssl-certificates)
  - [Software Installation](#software-installation)
  - [Outbound Configuration](#outbound-configuration)
  - [Basic Outbound Testing](#basic-outbound-testing)
  - [Inbound Configuration](#inbound-configuration)
  - [Local Relay Testing](#local-relay-testing)
  - [Allowing Remote Access](#allowing-remote-access)
  - [Configuring mailx](#configuring-mailx)


## Overview

When outbound network connections to ports 25 (_smtp_), 587 (_submission_), 465 (_smtps_) are blocked (as with AT\&T U-verse), sending email from a system can be troublesome; however it's likely that other outbound ports are open which can be used instead. This article will cover setting up an inbound SSL authenticated-only smtpd instance on port 443, which then relays email upstream to a standard mail provider also using SSL with the outbound connection requiring authentication.

```
any client software -> this.server:443 w/SSL and auth -> upstream normal SMTP w/SSL and auth
```


## Requirements

  - [CentOS 7](http://centos.org/)
  - [Postfix MTA](http://www.postfix.org/)
  - [Cyrus SASL](http://asg.web.cmu.edu/sasl/)
  - [OpenSSL](http://www.openssl.org/)
  - [Heirloom mailx](http://heirloom.sourceforge.net/mailx.html)
  - [iptables](http://en.wikipedia.org/wiki/Iptables)

Note: any modern Linux distribution should work, however package installation names and config file locations may vary slightly. CentOS 7 ships with Postfix 2.10.x, Cyrus SASL 2.1.x and OpenSSL 1.0.2k.


## Conventions Used

  - **mx.mydomain.org** - the name of the DNS listed server being implemented here
  - **supercoolusername**/**supercoolpassword** - the login created for _mx.mydomain.org_ herein
  - **smtp.mailgun.org** - we will use [Mailgun](http://www.mailgun.com/) as the upstream hop, any upstream standard SMTP will work
  - **postmaster@mydomain.org**/**mgdomainpass** - the login credentials to the upstream Mailgun server
  - **port 443** - port 443/tcp is normally used for HTTPS and tends to be open in all firewalls, etc.

Note: any port can be used depending on needs, 443 was chosen as it's already open in most firewalls and other network blocking situations. This would mean you cannot share this SMTP relay server with a true HTTPS website, however. Using port 60 is a good alternate choice if port 443 is not possible.


## SSL Certificates

Both inbound to the relay server and outbound to the upstream host connections will be implemented - obtain an SSL certificate with the name of your preferred domain. This can be self-signed, a free one from [CAcert](https://cacert.org), [StartCom SSL](https://www.startssl.com/), etc. - it's the same type of SSL key/certificate used in Apache webservers.

You will typically need the Intermediate CA certificates for your SSL cert as appropriate and the Intermediate for the upstream SMTP server. Intermediate CA certificates are generally used to prevent man-in-the-middle attacks on SSL connections and provide a higher security posture.


## Software Installation

Install the required software:

```
yum -y install postfix cyrus-sasl cyrus-sasl-lib cyrus-sasl-plain cyrus-sasl-md5 openssl mailx iptables-services
```


## Outbound Configuration

Configuring the outbound SMTP from the server first is recommended; it's easier to test locally from the server before setting up the inbound SMTP daemon. By configuring and testing this portion first we confirm any future failures should not be related to dispatching outbound email to the upstream connection.

First, obtain the upstream Intermediate CA certificate(s) and save them concatenated to a single PEM in `/etc/postfix/`. The Mailgun services herein use Geotrust Global CA so we'll grab that:

```
cd /etc/postfix/
wget https://www.geotrust.com/resources/root_certificates/certificates/GeoTrust_Global_CA.pem
```

Next, create a plain text file `/etc/postfix/sasl_passwd`that contains the upstream authentication username/password info. Secure it from prying eyes and then use `postmap` to create a Postfix-friendly database file out of it.

> For Mailgun this information is listed separately for each one of your configured domains, **not** your primary account login; the username is typically _postmaster@domain.name_ in form.

```
echo 'smtp.mailgun.org postmaster@mydomain.org:mgdomainpass' >> /etc/postfix/sasl_passwd
postmap /etc/postfix/sasl_passwd
chmod 0640 /etc/postfix/sasl_passwd*
chown root:postfix /etc/postfix/sasl_passwd*
```

Now add the configuration to Postfix for the outbound connection - these are the `smtp_*` settings (think "postfix as a client"):

```
/etc/postfix/main.cf

myhostname = mx.mydomain.org
mynetworks_style = host
relayhost = [smtp.mailgun.org]:587
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_sasl_security_options = noanonymous
smtp_tls_security_level = may
smtp_tls_note_starttls_offer = yes
smtp_tls_CAfile = /etc/postfix/GeoTrust_Global_CA.pem
```

Lastly ensure Postfix is enabled at boot and start it:

```
systemctl enable --now postfix.service
```

> If this is the first time configuring Postfix on a new installation, be sure and set the other needed settings such as `inet_interfaces`, `mynetworks_style` and similar configuration options. By default the config is locked to localhost and cannot receive mail from the outside world.


## Basic Outbound Testing

At this point you should be able to send an email using `mail` which will connect to the local Postfix instance on port 25 and relay your outbound email:

```
mail -s "Test subject" myemail@gmail.com
```

Examining `/var/log/maillog` should show the email being accepted by the upstream host for delivery like so:

```
/var/log/maillog

Oct  4 18:00:53 myserver postfix/smtp[949]: 8C3F4421E2: to=<myemail@gmail.com>, relay=smtp.mailgun.org[50.56.21.178]:587, delay=0.95, delays=0.35/0.11/0.28/0.21, dsn=2.0.0, status=sent (250 Great success)
```

Before proceeding further, ensure that this basic functionality is working.


## Inbound Configuration

This part is a little more involved, as precautions need to be taken to ensure that SSL is mandatory and a username/password is required for anything. Only persons with these credentials can send email, even to local server users.

First, save the SSL key and certificate for this server together in a single PEM file; additionally you'll need the Intermediate CA from where the certificate was obtained. StartCom SSL is used in this example.

> The key must be decrypted (without a password) - you may have to manually decrypt the key first, like so: `openssl rsa -in` _encrypted.key_ `-out` _decrypted.key_

```
cd /etc/postfix/
wget -O StartCom_CA.pem http://www.startssl.com/certs/sub.class1.server.ca.pem
cat /path/to/server.key >> /etc/postfix/mydomain-org.pem
cat /path/to/server.crt >> /etc/postfix/mydomain-org.pem
chmod 0640 /etc/postfix/mydomain-org.pem
chown root:postfix /etc/postfix/mydomain-org.pem
```

Next, create a SASL database file with the user/pass combo of your choosing to use for logging in -- the `-c` flag is to create a new databases, so if you're adding a second, third, etc. user drop that from the command. The `-u foo` option is the _realm_ that is used for authentication domain later, it can be anything but we'll use our DNS name _mx.mydomain.org_ to make it easy.

```
saslpasswd2 -c -f /etc/postfix/sasldb2 -u mx.mydomain.org -a smtpauth supercoolusername
chmod 0640 /etc/postfix/sasldb2
chown root:postfix /etc/postfix/sasldb2
```

Update the Postfix SASL config file to use this database file instead of the default which is [PAM](http://en.wikipedia.org/wiki/Pluggable_authentication_module) -- we're not going to create system users at all. This also means there is no need to run the `saslauthd` daemon, the SASL library dynamically linked into Postfix (`/usr/libexec/postfix/smtpd` -\> `/usr/lib64/libsasl2.so.2`) will read this config and use the DB directly as needed.

```
/etc/sasl2/smtpd.conf

pwcheck_method: auxprop
auxprop_plugin: sasldb
sasldb_path: /etc/postfix/sasldb2
mech_list: plain login
log_level: 1
```

Add the configuration of the `smtpd_*` settings in Postfix to accept the incoming SSL connections, again specifying parameters to lock it down for security. Note that the `smtpd_sasl_local_domain` setting matched the _realm_ named used above in the `saslpasswd2` command.

```
/etc/postfix/main.cf

smtpd_tls_cert_file = /etc/postfix/mydomain-org.pem
smtpd_tls_key_file = /etc/postfix/mydomain-org.pem
smtpd_tls_CAfile = /etc/postfix/StartCom_CA.pem
smtpd_tls_loglevel = 0
smtpd_tls_mandatory_ciphers = medium
smtpd_tls_mandatory_protocols = !SSLv2, !SSLv3
smtpd_sasl_type = cyrus
smtpd_sasl_local_domain = mx.mydomain.org
smtpd_sasl_path = smtpd
smtpd_sasl_security_options = noanonymous
broken_sasl_auth_clients = yes
```

The last major step is to define the listening daemon on a port; the `/etc/postfix/master.cf` file usually contains a commented out example for **smtps** (port 465), this definition should be inserted right at that spot. Because we want the standard localhost:25 connection to still work for basic server things, we're explicitly passing some of the "force SSL, force login" settings here as commandline options only on this port. The **443** is the port to use, change as you see fit.

```
/etc/postfix/master.cf

443       inet  n       -       n       -       -       smtpd
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_tls_security_level=encrypt
  -o smtpd_tls_auth_only=yes
  -o smtpd_enforce_tls=yes
  -o smtpd_client_restrictions=permit_sasl_authenticated,reject
  -o smtpd_recipient_restrictions=permit_sasl_authenticated,reject_unauth_destination
  -o milter_macro_daemon_name=ORIGINATING
```

Finally, restart Postfix:

```
systemctl restart postfix
```


## Local Relay Testing

Before configuring iptables to allow traffic in, testing should be performed. First you need to generate a base64 encoded version of the username and password to be used as a SMTP client, this is required for manual testing and is part of the PLAIN style SMTP specification.

```
echo -ne '\000supercoolusername\000supercoolpassword' | openssl base64
```

This should give a string of alphanumeric characters and maybe a few symbols that is used in the `AUTH PLAIN` command typed below. then try testing the process locally using the `openssl` tool. Things you type are denoted with `### TYPE THIS ###` inline - for more help on how to manually "talk SMTP" hit up Google.

```
# openssl s_client -quiet -starttls smtp -connect localhost:443

250 DSN
EHLO example.com                      ### TYPE THIS ###
250-mx.mydomain.org
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-AUTH LOGIN PLAIN
250-AUTH=LOGIN PLAIN
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN
AUTH PLAIN XXXXXXXXXXXXXXXXXXXX       ### TYPE THIS ###
235 2.7.0 Authentication successful
MAIL FROM: someone@server.local       ### TYPE THIS ###
250 2.1.0 Ok
RCPT TO: myemail@gmail.com            ### TYPE THIS ###
250 2.1.5 Ok
QUIT                                  ### TYPE THIS ###
221 2.0.0 Bye
```

The above is actually testing multiple things all at once; TLS (SSL) is working, authentication is working and relaying as an authenticated user is working.

Next, test that **not** using SSL is blocked:

```
# telnet localhost 443

Trying ::1...
Connected to localhost.
Escape character is '^]'.
220 mx.mydomain.org ESMTP Postfix
EHLO example.com
250-mx.mydomain.org
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-STARTTLS
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN
AUTH PLAIN XXXXXXXXXXXXXXXXXXXX
530 5.7.0 Must issue a STARTTLS command first
QUIT
221 2.0.0 Bye
```

Finally, test that you cannot try sending mail without authentication - the `RCPT TO:` should be tested against local server accounts (i.e. root@localhost) and relayed accounts (i.e. myemail@gmail.com). In all cases the **554** error should happen:

```
# openssl s_client -quiet -starttls smtp -connect localhost:443

250 DSN
EHLO example.com
250-mx.mydomain.org
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-AUTH LOGIN PLAIN
250-AUTH=LOGIN PLAIN
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN
MAIL FROM: someone@server.local
250 2.1.0 Ok
RCPT TO: myemail@gmail.com
554 5.7.1 <localhost[::1]>: Client host rejected: Access denied
QUIT
221 2.0.0 Bye
```

All of these tests ensure that it's working when SSL and authentication is used, but denied in all other cases. We specifically did not enable the local server to send email **on this port** without SSL and authentication, however localhost:25 should still work for that.


## Allowing Remote Access

As a last step, open the iptables firewall to allow port 443 usage.

> Do not open port 25 unless your server actually receives normal email from the Internet

If you already use iptables, the rule might be added like so:

```
-A INPUT -m state --state NEW -m tcp -p tcp --dport 443 -j ACCEPT
```

If this is a brand new setup, or iptables wasn't used here's a more full-fledged example that allows 22 (SSH), 80 (HTTP) and 443 which is pretty average for a server (note: I recommend [fail2ban](http://www.fail2ban.org/wiki/index.php/Main_Page) if opening SSH):

```
/etc/sysconfig/iptables

*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -p tcp -m tcp --dport 443 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 80 -j ACCEPT
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
-A INPUT -j REJECT --reject-with icmp-host-prohibited
-A FORWARD -j REJECT --reject-with icmp-host-prohibited
COMMIT

/etc/sysconfig/ip6tables

*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -p tcp -m tcp --dport 80 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 443 -j ACCEPT
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p ipv6-icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
-A INPUT -d fe80::/64 -p udp -m udp --dport 546 -m state --state NEW -j ACCEPT
-A INPUT -j REJECT --reject-with icmp6-adm-prohibited
-A FORWARD -j REJECT --reject-with icmp6-adm-prohibited
COMMIT
```

Once this is open, perform the same testing as the Local section above, except use the DNS name of the server to connect.

For completeness sake because we love security and stopping the bad guys, here's a simple `fail2ban` configuration that will monitor the SSH port for brute force attacks and null route the evildoers:

```
/etc/fail2ban/jail.local

[DEFAULT]
ignoreip = 127.0.0.1/8
bantime = 600
findtime  = 600
maxretry = 3

[sshd]
enabled = true
maxretry = 5
usedns = no
```


## Configuring mailx

One of the most common ways to send email from cron jobs, scripts and other assorted commands is using the `mail` (_mailx_) command. To configure the client systems to use this relay, a `~/.mailrc` configuration needs to be set up like so:

```
~/.mailrc

account myrelay {
  set from=myname@mylaptop.local
  set smtp-hostname=mylaptop.local
  set smtp=smtp://supercoolusername:supercoolpassword@mx.mydomain.org:443
  set smtp-use-starttls
  set smtp-auth=plain
  set ssl-verify=warn
  set v15-compat
}
```

Then the usage is with the `-A myrelay` commandline option:

```
mail -A myrelay -s "Test email" myemail@gmail.com
```

Using other clients should work in a normal fashion with their built in configuration options, just choose `STARTTLS` in the connection options (not **direct SSL** or similar). The use of direct SSL (_smtps_) to the server is not used herein; `STARTTLS` is more universally compatible.