Süsteemihaldus 2017/18 kevad

Should work ... if you find any errors / comments they are welcome at alo.peets@ut.ee

0.Make sure you did finish all the tasks of the previous weeks.

We expect here all personal domains are configured, VM is referable by its FDQN internally and externally from UT network (172.17.0.0/16 and 193.40.154.0/23) as well as from client machines ( over VPN ).

PS! Before you continue lab instructors highly recommend to read a very good manual about how e-mail works and how to set it up, from Christoph Haas https://workaround.org/ispmail/stretch page.

1.Setting up personal mail server

Now our fake DNS domain est is ready and populated with individual sub-domains inside, VMs are reachable by FDQN. Moreover each student has full control of his own domain, which means we can add as many names as we wish into our domain. Ideally each service has conventional names, like www for web, mail for smtp or ftp for FTP server. As we start here with e-mail service, let's add the mail host name as well as Mail Exchanger pointer to our DNS configuration.

Understanding MX Records

DNS type MX (Mail eXchange) records are used to specify the mail server(s) responsible for accepting e-mail messages on behalf of an entire domain. If there is no MX record for the domain, type A record will be queried for instead. If there is no A record either, it is not possible to send e-mails to the domain. Our goal is to set up the MX records so that the e-mails for the <vm_name>.est domain will be directed to mail.<vm_name>.est, which in turn points to our IP. So you will need to create a new type A record, too. If AAAA IPv6 records are present the postfix will use them by default, since we have no IPv6 setup we will not use AAAA records.

Here are some extra materials for type MX records. The minimum you will need to know is: what are the priority values of MX records and how to set them in the zone file.

• MX Record
• Wikipedia: MX Record

To test, query for the A and MX records you have just created (use either VM itself or the client machine):

The query should give the answer like (works only if you have set your DNS servers to be the Lab's ones):

teacher@teacher:~$ dig student-test.est -t MX

; <<>> DiG 9.10.3-P4-Debian <<>> -t mx student-test.est
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 45912
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 3

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;student-test.est.		IN	MX

;; ANSWER SECTION:
student-test.est.	900	IN	MX	10 mail.student-test.est.

;; AUTHORITY SECTION:
student-test.est.	900	IN	NS	ns1.student-test.est.

;; ADDITIONAL SECTION:
mail.student-test.est.	900	IN	A	193.40.155.0
ns1.student-test.est.	900	IN	A	193.40.155.0

;; Query time: 1 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Sun Mar 24 20:55:38 UTC 2019
;; MSG SIZE  rcvd: 116

Setting up NTP

Before we setup the SMTP service let's make sure our VM's time is in sync with the world time. As the mail delivery system stamps each mail with sent-out date and delivery-date it is very important that all servers have the timezone specified and the clock kept in sync. The Network Time Protocol (NTP) is used for that purpose, the details you may read here.

In Debian the NTP is handled by the package called ntp , so let's install it.

Understanding ntp client configuration

By default the ntp package comes with the list of predefined servers to sync against the following serves (according to /etc/ntp.conf :

...
pool 0.debian.pool.ntp.org iburst
pool 1.debian.pool.ntp.org iburst
pool 2.debian.pool.ntp.org iburst
pool 3.debian.pool.ntp.org iburst
...

Let's make sure our VMs do also rely on university's NTP server:

• ntp.ut.ee

Prepare for VM for mail service related packages

For the mail delivery services will want the hostname -f to return the FQDN of the VM. One option is to keep the line in /etc/hosts file:

172.17.64.X yourhost.yourdomain.est yourhost

... we did this in the beginning.

Now we have our DNS fully functional and therefore there is no point of keeping these line in /etc/hosts as the local DNS should resolve it anyway. Moreover there is more intelligent way to keep the FQDN information. If you specify the FQDN into /etc/hostname explicitly - the OS will be always provisioned with its FQDN irregardless of the entries in /etc/hosts and the Zone records in DNS configuration.

If it does not:

Generating TLS/SSL Certificates

In this lab you will need programs that are not installed by default. Start by installing Postfix, Dovecot, SpamAssassin and Alpine packages. As we mentioned during the first lab session there are other package management tools. apt can be very handy when you know exact package names and don't expect any complicated troubles with dependencies.

We will use SMTP and IMAP based mechanisms to exchange messages and to access mailboxes. In "plain" versions, these mechanisms send user names and passwords in clear text across the Internet. In order to protect sensitive data, we will enable support TLS (Transport Layer Security), which will encrypt the traffic. The "plain" versions will still be used by default, so we need to enable TLS on the client-side, too. The predecessor for TLS was called SSL (Secure Sockets Layer).

First we will need a TLS certificate that will be used to identify our servers and start the encrypted connection. Certificate file contains a public key and the corresponding private key is stored in separate file. Our certificates will be self-signed, meaning that there will be no Certification Authority to prove our identity. Some client software will warn about this. (We will fix this later in Apace lab.)

We will use the same (identical) certificate for both SMTP (Postfix) and IMAP (Dovecot) server. It simplifies configuration and authentication slightly in the beginning. In Debian system there is a recommended location for all the certificates and the private keys:

Setting Up SMTP Service Using the Postfix Software

First you need to install the requiered software

Postfix is an SMTP server (Mail Transfer Agent) that is relatively easy to configure. However, you must still read and understand the "Postfix Configuration- Basics" document before you proceed.

• Postfix
• Postfix Configuration- Basics

The locations of the Postfix configuration and log file:

• Configuration file: /etc/postfix/main.cf
• Log file for e-mail related log messages: /var/log/mail.log
• also, keep on monitoring the /var/log/messages log file

In postfix the $mydomain parameter specifies the parent domain of $myhostname. By default, it is derived from $myhostname by stripping off the first part (unless the result would be a top-level domain).

Canonical sender aliases

By default the the postfix will supplement FDQN of the machine it is running in From/To: field's host part. For example if user teacher is sending out and receiving mail from host05.teacher.est machine - the From: will look like:

• teacher@host05.teacher.est

The canonical email addresses however should contain only the domain information after the @ sign. Therefore we have to force the postfix to remove the undesired host information from the From field when sending out. This is done using canonical sender alias maps (or canonical table). According the documentation, the canonical map should contain the rewrite rules, as follows:

<original> <alias>

Here original is the original is the <real user>@<sender host> and the alias is what should be shown instead in the from field. For example if we would need to hide the real user teacher behind the alias SA.Teacher when sending out, the canonical map should contain following:

teacher@host05.teacher.est SA.Teacher@teacher.est

The postfix would rewrite the From field correspondingly for each mail that is sent out by teacher user.

In our setup we first of all will focus on rewriting the host part of the email addresses specified in From field. And the user names we take later (will hide the using both virtual and local aliases). In case only the host part needs to be replaced the canonical file will look like:

@host05.teacher.est @teacher.est

And with this setup any user sending an email from host host05.teacher.est will have his From field rewritten with @teacher.est (leaving his user name untouched).

Now add the configuration to postfix telling to use canonical map

If everything is Ok, you can start up the Postfix server. SMTP port will need to be opened from the firewall, too.

Mail server can now be tested by sending mails from/to the different user accounts on your machine.

• You can add new user account using following command:
  • # adduser userNameHere

Mail Aliases

Mail aliases are used to forward all e-mails sent to some address to some other address. Using aliases, you can also create an e-mail address that is not in strict connection with any of the system's user accounts. Aliases are configured in the /etc/aliases file. You can read more about aliases and its file format with command $ man aliases .

If you might have noticed alpine can autocomplete e-mail addresses so you could just write username and it will automatically add your hostname to complete the address. The problem here is that you don't want to send out e-mails with your hostname in the header (i.e X-Original-To: aliastest@host06.teacher.est).In each users' home folder there is alpine configuration file (i.e /home/user/.pinerc ), but editing it will make changes only to particular user alpine environment. Therefore we must create centralized an alpine configuration file and modify it to achieve our goal.

General check of the Postfix configuration

The following commands are useful to review the active configuration. Here active configuration means - the configuration from main.cf loaded once service is restarted.

IMAP Service Using Dovecot Software

We did use a "local mailbox" model for testing the e-mail service, but nowadays most of the e-mail reading is done over the network protocols such as IMAP and POP3. Actually, web-based solutions are the most popular by far, but even these often use IMAP behind the scenes.

We will now set up an easy-to-configure IMAP server called Dovecot (the package should be installed already).

Again, you should read and understand the configuration part of the Dovecot manual. Dovecot Wiki. Main configuration file for Dovecot is /etc/dovecot/dovecot.conf and because of the modular design each module has separate configuration file under /etc/dovecot/conf.d directory.
Details about Dovecot configuration files:
Dovecot Wiki

Configuring dovecot logging module

Configuring dovecot ssl module

Configuring dovecot authentication module

The simplest authentication mechanism is PLAIN. The client simply sends the password unencrypted to Dovecot. All clients support the PLAIN mechanism, but obviously there's the problem that anyone listening on the network can steal the password. For that reason (and some others) other mechanisms were implemented.

Today however many people use SSL/TLS, and there's no problem with sending unencrypted password inside SSL secured connections. So if you're using SSL, you probably don't need to bother worrying about anything else than the PLAIN mechanism.

Another plaintext mechanism is LOGIN. It's typically used only by SMTP servers to let Outlook clients perform SMTP authentication. Note that LOGIN mechanism is not the same as IMAP's LOGIN command. The LOGIN command is internally handled using PLAIN mechanism.

Configuring dovecot mail module

General check of the Dovecot configuration

Firewall Configuration

If everything was set up properly the dovecot service should now listen for 2 port in TCP stack. What are these ports ?

If you know the answer, declare the ports in firewall configuration:

Thunderbird configuration

In order to verify Dovecot functionality we can use popular e-mail client called Thunderbird https://www.mozilla.org/en-US/thunderbird/.

• In Linux hosts you can just install the corresponding package using package manager

SMTP Authentication

You may have noticed while configuring the Thunderbird client, that the SMTP client was configured automatically along with IMAP client. The mail sending (submission) from Thunderbird is still partial as the SMTP is not authenticated by default as opposed to IMAP. The disadvantage of the unauthenticated SMTP is that it only allows the SMTP client to submit mails to the same domain the SMTP server is serving for. The cross-domain submission will be however impossible.

You may test it as follows:

Example in teacher.est domain:

• teacher and teacher2 accounts were in use
• email sent from teacher2@teacher.est to teacher@teacher.est The message header was as follows:

...
From: teacher2@teacher.est
To: teacher@teacher.est
Subject: Test SMTP
...

Now destination user should have received the email (you may check it using the alpine command of the destination user, or alternatively you may setup another IMAP/SMTP client in Thunderbird.

Now if you try to address mail to the destination in different domain, the SMTP server will respond with a message:

Let's fix the error by adding the authentication to SMTP, like we did with the IMAP:

• enabling STARTTLS and explicit TLS
• enabling PLAIN authentication only over encrypted channel

The differences are however in authentication backend:

• Dovecot relies by default on system PAM when authenticating clients, therefore all the user accounts in the host do have IMAP accounts automatically.
  • The other authentication back-ends possible for Dovecot are: Database, LDAP, Plain password file, etc.
• Postifx suggests using SASL protocol for authentication and the system PAM doesn't suggest any SASL implementation to use by default (should be configured separately by installing and configuring Cyrus SASL for example).
  • Alternatively a SASL implementation of the Dovecot may be used (in this case we do not have to install any additional software)
  • In this case however the SASL needs to be enabled in the Dovecot

The SMTP authentication chain looks like:

• SMTP client -> SMTP server (postfix) -> Dovecot SASL -> System PAM

Let's start step by step enabling these features:

Configure Dovecot enabling SASL protocol

# Postfix smtp-auth
#unix_listener /var/spool/postfix/private/auth {
#  mode = 0666
#}

Configure Postfix to use Dovecot's SASL back-end

smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
smtpd_sasl_auth_enable = yes

Configure Postfix to enable submission port (587) for MUAs

SMTP communication between mail servers uses port 25. Mail clients on the other hand, often submit the outgoing emails to a mail server on port 587. Despite being deprecated, mail providers sometimes still permit the use of historic nonstandard port 465 for this purpose (i.e University of Tartu).

First let's enable the submission port with SASL authentication over STARTTLS. We also allow some certain limitations on email submission:

• Disable anonymous submission noanonymous
• Restrict SMTP clients: only allow SASL authenticated
• Restrict Relaying: only allow SASL authenticated
• Restrict Originators: From may only contain the "domain" of the SMTP server
  • SMTP client cannot put random values into From field

#submission inet n       -       -       -       -       smtpd
#  -o syslog_name=postfix/submission
#  -o smtpd_tls_security_level=encrypt
#  -o smtpd_sasl_auth_enable=yes
#  -o smtpd_reject_unlisted_recipient=no
#  -o smtpd_client_restrictions=$mua_client_restrictions
#  -o smtpd_helo_restrictions=$mua_helo_restrictions
#  -o smtpd_sender_restrictions=$mua_sender_restrictions
#  -o smtpd_recipient_restrictions=
#  -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
#  -o milter_macro_daemon_name=ORIGINATING

Now let's also enable secure submission over explicit TLS (SMTPS). We will use the same limitation policies as in submission block

#smtps     inet  n       -       -       -       -       smtpd
#  -o syslog_name=postfix/smtps
#  -o smtpd_tls_wrappermode=yes
#  -o smtpd_sasl_auth_enable=yes
#  -o smtpd_reject_unlisted_recipient=no
#  -o smtpd_client_restrictions=$mua_client_restrictions
#  -o smtpd_helo_restrictions=$mua_helo_restrictions
#  -o smtpd_sender_restrictions=$mua_sender_restrictions
#  -o smtpd_recipient_restrictions=
#  -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
#  -o milter_macro_daemon_name=ORIGINATING

Enable the submission port in firewall

Testing

Spam Filtering with SpamAssassin and Procmail

Apache SpamAssassin is one of the best open source spam filters. It uses a robust scoring framework and plug-ins to integrate a wide range of advanced heuristic and statistical analysis tests on email headers and body text including text analysis, Bayesian filtering, DNS blocklists, and collaborative filtering databases. We will use SpamAssassin spamd/spamc model through procmail software. spamd is a SpamAssassin service that calculates "spam scores" for the messages. spamc is a client for the spamd. procmail is a rule-based mail processing tool. We will use procmail to do the real filtering based on the spam scores set by SpamAssassin.

You can read more about how to set up SpamAssassin in Debian here https://wiki.debian.org/DebianSpamAssassin .

Sometimes you need to modify SpamAssassin and its rules to customize its action. You can read more aboud how scores are calculated from here. https://www.clickz.com/understand-spamassassin-for-better-delivery-rates/69157/ Following example has been taken from http://www.informit.com/articles/article.aspx?p=375704&seqNum=6 with modifications but it has some more explanation of how it works.

body MY_SPAMASSASSIN_RULE        /this is a test text for custom spamassassin rule/is
describe MY_SPAMASSASSIN_RULE    Test of my spamassassin rule
score MY_SPAMASSASSIN_RULE       10

Add auto-reply for tester account

If everything worked till now lets make it easy for our automated tester scripts to white-box test your e-mail server functionality.

LOGFILE=/home/tester/procmailrc.log
VERBOSE=on
#Autoreply for every incoming mail
:0 c
* !^FROM_DAEMON
* !^X-Loop: teacher@teacher.est
| (formail -rk -A "Precedence: junk" -A "X-Loop: teacher@teacher.est" ; echo "E-mail received at : " `date`) | /usr/sbin/sendmail -t -oi -oe