Süsteemihaldus 2019/20 kevad

0.Make sure you did finish all the tasks of the previous weeks.

We expect here all personal domains are configured, VM is referable by its FDQN internally and externally from UT network (172.17.0.0/16) as well as from over VPN.

PS! There's a short example of everything we are going to do today: https://www.google.com/search?client=firefox-b-d&q=centos+email+guide.

1.Setting up personal mail server

Now our DNS domain sa.cs.ut.ee is ready and populated with individual sub-domains inside, VMs are reachable by FDQN. Moreover each student has full control of his own domain, which means we can add as many names as we wish into our domain. Ideally each service has conventional names, like www for web, mail for smtp or ftp for FTP server. As we start here with e-mail service, let's add the mail host name as well as Mail Exchanger pointer to our DNS configuration.

Understanding MX Records

DNS type MX (Mail eXchange) records are used to specify the mail server(s) responsible for accepting e-mail messages on behalf of an entire domain. If there is no MX record for the domain, type A record will be queried for instead. If there is no A record either, it is not possible to send e-mails to the domain. Our goal is to set up the MX records so that the e-mails for the <vm_name>.sa.cs.ut.ee domain will be directed to mail.<vm_name>.sa.cs.ut.ee, which in turn points to our IP. So you will need to create a new type A record, too. If AAAA IPv6 records are present the postfix will use them by default, since we have no IPv6 setup we will not use AAAA records.

Here are some extra materials for type MX records. The minimum you will need to know is: what are the priority values of MX records and how to set them in the zone file.

• MX Record
• Wikipedia: MX Record

To test, query for the A and MX records you have just created:

The query should give the answer like (works only if you have set your DNS servers to be the Lab's ones):

[root@scoringserver ~]# dig scoring.sa.cs.ut.ee -t MX

; <<>> DiG 9.9.4-RedHat-9.9.4-73.el7_6 <<>> scoring.sa.cs.ut.ee -t MX
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 6694
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 1, ADDITIONAL: 2

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;scoring.sa.cs.ut.ee.		IN	MX

;; ANSWER SECTION:
scoring.sa.cs.ut.ee.	604800	IN	CNAME	sa.cs.ut.ee.
sa.cs.ut.ee.		604800	IN	MX	10 mail.scoring.sa.cs.ut.ee.

;; AUTHORITY SECTION:
sa.cs.ut.ee.		604800	IN	NS	ns.sa.cs.ut.ee.

;; ADDITIONAL SECTION:
ns.sa.cs.ut.ee.		604800	IN	A	172.17.66.134

;; Query time: 1 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: E apr   06 05:56:23 EEST 2020
;; MSG SIZE  rcvd: 116

Setting up NTP

Before we setup the SMTP service let's make sure our VM's time is in sync with the world time. As the mail delivery system stamps each mail with sent-out date and delivery-date it is very important that all servers have the timezone specified and the clock kept in sync. The Network Time Protocol (NTP) is used for that purpose, the details you may read here.

In CentOS 8 the NTP is handled by the package called chrony , so let's install it.

Understanding ntp client configuration

By default the chrony package comes with the list of predefined servers to sync against the following serves (according to /etc/chrony.conf :

...
pool 2.centos.pool.ntp.org iburst
...

Let's make sure our VMs do also rely on university's NTP server:

• ntp.ut.ee

Prepare for VM for mail service related packages

For the mail delivery services will want the hostname -f to return the FQDN of the VM. One option is to keep the line in /etc/hosts file:

<external_IP> <vm_name>.sa.cs.ut.ee <vm_name>

... we did this in the beginning.

Now we have our DNS fully functional and therefore there is no point of keeping these line in /etc/hosts as the local DNS should resolve it anyway. Moreover there is more intelligent way to keep the FQDN information. If you specify the FQDN into /etc/hostname explicitly - the OS will be always provisioned with its FQDN regardless of the entries in /etc/hosts and the Zone records in DNS configuration.

If it does not:

Setting up software

In this lab you will need programs that are not installed by default. Start by installing Postfix, Dovecot, SpamAssassin and Alpine packages. As we mentioned during the first lab session there are other package management tools. dnf can be very handy when you know exact package names and don't expect any complicated troubles with dependencies.

You can use dnf search to find package names.

We will use SMTP and IMAP based mechanisms to exchange messages and to access mailboxes. In "plain" versions, these mechanisms send user names and passwords in clear text across the Internet. We will first set up our system in "plain" mode, and in next lab we will secure your installations with TLS (Transport Layer Security), which will encrypt the traffic.

Setting Up SMTP Service Using the Postfix Software

Postfix is an SMTP server (Mail Transfer Agent) that is relatively easy to configure. However, you must still read and understand the "Postfix Configuration- Basics" document before you proceed.

• Postfix
• Postfix Configuration- Basics

The locations of the Postfix configuration and log file:

• Configuration file: /etc/postfix/main.cf
• Log file for e-mail related log messages: /var/log/maillog
• also, keep on monitoring the /var/log/messages log file

In postfix the $mydomain parameter specifies the parent domain of $myhostname. By default, it is derived from $myhostname by stripping off the first part (unless the result would be a top-level domain).

Canonical sender aliases

By default the the postfix will supplement FDQN of the machine it is running in From/To: field's host part. For example if user teacher is sending out and receiving mail from host05.teacher.sa.cs.ut.ee machine - the From: will look like:

• teacher@host05.teacher.sa.cs.ut.ee

The canonical email addresses however should contain only the domain information after the @ sign. Therefore we have to force the postfix to remove the undesired host information from the From field when sending out. This is done using canonical sender alias maps (or canonical table). According the documentation, the canonical map should contain the rewrite rules, as follows:

<original> <alias>

Here original is the original is the <real user>@<sender host> and the alias is what should be shown instead in the from field. For example if we would need to hide the real user teacher behind the alias SA.Teacher when sending out, the canonical map should contain following:

teacher@host05.teacher.sa.cs.ut.ee SA.Teacher@teacher.sa.cs.ut.ee

The postfix would rewrite the From field correspondingly for each mail that is sent out by teacher user.

In our setup we first of all will focus on rewriting the host part of the email addresses specified in From field. And the user names we take later (will hide the using both virtual and local aliases). In case only the host part needs to be replaced the canonical file will look like:

@host05.teacher.sa.cs.ut.ee @teacher.sa.cs.ut.ee

And with this setup any user sending an email from host host05.teacher.sa.cs.ut.ee will have his From field rewritten with @teacher.sa.cs.ut.ee (leaving his user name untouched).

Now add the configuration to postfix telling to use canonical map

If everything is Ok, you can start up the Postfix server. If you did not specify allow-all when creating the VM, SMTP port will need to be opened from the firewall on minu.etais.ee, too.

Do also open ports for service smtp with firewall-cmd .

Mail server can now be tested by sending mails from/to the different user accounts on your machine.

• You can add new user account using following command:
  • # adduser newUserName

Mail Aliases

Mail aliases are used to forward all e-mails sent to some address to some other address. Using aliases, you can also create an e-mail address that is not in strict connection with any of the system's user accounts. Aliases are configured in the /etc/aliases file. You can read more about aliases and its file format with command $ man aliases .

If you might have noticed alpine can autocomplete e-mail addresses so you could just write username and it will automatically add your hostname to complete the address. The problem here is that you don't want to send out e-mails with your hostname in the header (i.e X-Original-To: aliastest@host06.teacher.sa.cs.ut.ee). In each users' home folder there is alpine configuration file (i.e /home/<user>/.pinerc ), but editing it will make changes only to particular user alpine environment. Therefore we must create centralized an alpine configuration file and modify it to achieve our goal.

General check of the Postfix configuration

The following commands are useful to review the active configuration. Here active configuration means - the configuration from main.cf loaded once service is restarted.

IMAP Service Using Dovecot Software

We did use a "local mailbox" model for testing the e-mail service, but nowadays most of the e-mail reading is done over the network protocols such as IMAP and POP3. Actually, web-based solutions are the most popular by far, but these also usually use IMAP behind the scenes.

We will now set up an easy-to-configure IMAP server called Dovecot (the package should be installed already).

Again, you should read and understand the configuration part of the Dovecot manual. Dovecot Wiki. Main configuration file for Dovecot is /etc/dovecot/dovecot.conf and because of the modular design each module has separate configuration file under /etc/dovecot/conf.d directory.
Details about Dovecot configuration files:
Dovecot Wiki

Configuring dovecot logging module

Configuring dovecot authentication module

The simplest authentication mechanism is PLAIN. The client simply sends the password unencrypted to Dovecot. All clients support the PLAIN mechanism, but obviously there's the problem that anyone listening on the network can steal the password. For that reason (and some others) other mechanisms were implemented. But this week, for the sake of learning, we will set it up without any security mechanisms.

NB! Do not reuse any passwords you already have used somewhere else - they can be sniffed and found from the network, as we will set up systems without TLS.

Configuring dovecot mail module

General check of the Dovecot configuration

Firewall Configuration

If everything was set up properly the dovecot service should now listen for 2 port in TCP stack. What are these ports ?

Declare the ports in firewall configuration:

Thunderbird configuration

In order to verify Dovecot functionality we can use popular e-mail client called Thunderbird https://www.mozilla.org/en-US/thunderbird/.

• In Linux hosts you can just install the corresponding package using package manager

SMTP Authentication

You may have noticed while configuring the Thunderbird client, that the SMTP client was configured automatically along with IMAP client. The mail sending (submission) from Thunderbird is still partial as the SMTP is not authenticated by default as opposed to IMAP. The disadvantage of the unauthenticated SMTP is that it only allows the SMTP client to submit mails to the same domain the SMTP server is serving for. The cross-domain submission will be however impossible.

You may test it as follows:

Example in teacher.sa.cs.ut.ee domain:

• teacher and teacher2 accounts were in use
• email sent from teacher2@teacher.sa.cs.ut.ee to teacher@teacher.sa.cs.ut.ee The message header was as follows:

...
From: teacher2@teacher.sa.cs.ut.ee
To: teacher@teacher.sa.cs.ut.ee
Subject: Test SMTP
...

Now destination user should have received the email (you may check it using the alpine command of the destination user, or alternatively you may setup another IMAP/SMTP client in Thunderbird.

Now if you try to address mail to the destination in different domain, the SMTP server will respond with a message:

Let's fix the error by adding the authentication to SMTP, like we did with the IMAP, by enabling PLAIN authentication.

The differences are however in authentication backend:

• Dovecot relies by default on system PAM when authenticating clients, therefore all the user accounts in the host do have IMAP accounts automatically.
  • The other authentication back-ends possible for Dovecot are: Database, LDAP, Plain password file, etc.
• Postfix suggests using SASL protocol for authentication and the system PAM doesn't suggest any SASL implementation to use by default (should be configured separately by installing and configuring Cyrus SASL for example).
  • Alternatively a SASL implementation of the Dovecot may be used (in this case we do not have to install any additional software)
  • In this case however the SASL needs to be enabled in the Dovecot

The SMTP authentication chain looks like:

• SMTP client -> SMTP server (postfix) -> Dovecot SASL -> System PAM

Let's start step by step enabling these features:

Configure Dovecot enabling SASL protocol

# Postfix smtp-auth
#unix_listener /var/spool/postfix/private/auth {
#  mode = 0666
#}

Configure Postfix to use Dovecot's SASL back-end

smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
smtpd_sasl_auth_enable = yes

Configure Postfix to enable submission port (587) for MUAs

SMTP communication between mail servers uses port 25. Mail clients on the other hand, often submit the outgoing emails to a mail server on port 587. Despite being deprecated, mail providers sometimes still permit the use of historic nonstandard port 465 for this purpose (i.e University of Tartu).

First let's enable the submission port with SASL authentication. We also allow some certain limitations on email submission:

• Disable anonymous submission noanonymous
• Restrict SMTP clients: only allow SASL authenticated
• Restrict Relaying: only allow SASL authenticated
• Restrict Originators: From may only contain the "domain" of the SMTP server
  • SMTP client cannot put random values into From field

#submission inet n       -       -       -       -       smtpd
#  -o syslog_name=postfix/submission
#  -o smtpd_sasl_auth_enable=yes
#  -o smtpd_reject_unlisted_recipient=no
#  -o smtpd_client_restrictions=$mua_client_restrictions
#  -o smtpd_helo_restrictions=$mua_helo_restrictions
#  -o smtpd_sender_restrictions=$mua_sender_restrictions
#  -o smtpd_recipient_restrictions=
#  -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
#  -o milter_macro_daemon_name=ORIGINATING

• Enable the submission block (uncomment the header line)
  • The rest of the options you may configure as in the submission block
    • -o smtpd_sasl_auth_enable=yes
    • -o smtpd_sasl_path=private/auth
    • -o smtpd_sasl_security_options=noanonymous
    • -o smtpd_client_restrictions=permit_sasl_authenticated,reject
    • -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
    • -o milter_macro_daemon_name=ORIGINATING
• Save the file
• Restart the postfix service

Enable the submission port in firewall

Testing

Spam Filtering with SpamAssassin and Procmail

Apache SpamAssassin is one of the best open source spam filters. It uses a robust scoring framework and plug-ins to integrate a wide range of advanced heuristic and statistical analysis tests on email headers and body text including text analysis, Bayesian filtering, DNS blocklists, and collaborative filtering databases. We will use SpamAssassin spamd/spamc model through procmail software. spamd is a SpamAssassin service that calculates "spam scores" for the messages. spamc is a client for the spamd. procmail is a rule-based mail processing tool. We will use procmail to do the real filtering based on the spam scores set by SpamAssassin.

You can read about SpamAssassin here: https://en.wikipedia.org/wiki/Apache_SpamAssassin

Sometimes you need to modify SpamAssassin and its rules to customize its action. You can read more aboud how scores are calculated from here. https://www.clickz.com/understand-spamassassin-for-better-delivery-rates/69157/ Following example has been taken from http://www.informit.com/articles/article.aspx?p=375704&seqNum=6 with modifications but it has some more explanation of how it works.

body MY_SPAMASSASSIN_RULE        /this is a test text for custom spamassassin rule/is
describe MY_SPAMASSASSIN_RULE    Test of my spamassassin rule
score MY_SPAMASSASSIN_RULE       10

Add auto-reply for scoring account

If everything worked till now lets make it easy for our automated scoring scripts to white-box test your e-mail server functionality.

LOGFILE=/home/scoring/procmailrc.log
VERBOSE=on
#Autoreply for every incoming mail
:0 c
* !^FROM_DAEMON
* !^X-Loop: teacher@teacher.sa.cs.ut.ee
| (formail -rk -A "Precedence: junk" -A "X-Loop: teacher@teacher.sa.cs.ut.ee" ; echo "E-mail received at : " `date`) | /usr/sbin/sendmail -t -oi -oe