Secure email exchange
How email exchange works
Simple Mail Transfer Protocol (SMTP) is used to transmit emails both between the user and email server and between the mail servers. However, email client software is commonly configured to use SMTP only for sending emails. In order to receive emails client software uses either IMAP or POP3.
IMAP protocol is commonly used nowadays as it provides support for multiple simultaneous clients. Therefore, with IMAP it is possible to view emails in both the smartphone and in a laptop without the emails being deleted from the server. However, POP3 is an older protocol that first fetches the mail and then deletes it from the server which means that by default mail can be downloaded to a single device.
One might wonder if SMTP, IMAP and POP3 always encrypt the email traffic in order to protect confidential information from leaking. Actually, all of these protocols support the transmission of data over SSL / TLS but this is an optional feature. Special ports are reserved for these protocols to transmit data over SSL / TLS. Secure SMTP is called SMTPS and it uses port 465. Secure IMAP is called IMAPS and it uses port 993. Secure POP3 is called POP3S and it uses port 995. It is obvious that the extra S in the abbreviations stands for secure and it just means that the protocol is used inside of a SSL / TLS session. Most email servers support the secure versions of these protocols but the user may have to configure the email client software to use the correct port.
There is a feature that enables encryption even for the plain SMTP, IMAP, POP3 and this feature is called STARTTLS. STARTTLS adds SSL / TLS by default if the corresponding mail server supports this. Most well known email providers have enabled support for STARTTLS. However, if a mail server does not support STARTTLS, then the user will have to configure the email client software to use a port that provides an encrypted connection.
Security and privacy of email exchange
The protocols that are used to exchange emails are not designed to take security and privacy seriously. These protocols were designed to get the job done in an environment where security was not so relevant. However, these protocols are still alive and it is difficult to modify them or to replace them with more secure protocols.
By default email is not encrypted, some links between the mail servers might be encrypted but this does not mean that the content of the email can not be intercepted, read and modified. It might be possible to encrypt the links between the mail clients and mail servers but while the mail is routed through the Internet it might pass servers that do not support encryption. In addition, as the content of the mail is not end-to-end encrypted, the mail servers are able to read the contents of the emails.
Some SMTP servers support STARTTLS which enables to automatically switch from an unencrypted connection to an encrypted connection over SSL / TLS. However, STARTTLS is mostly supported by the major email providers and many servers still do not provide this option. When STARTTLS is not available and the mail client is not forced to use a secure mail exchange protocol then the email is exchanged in plaintext, i.e., in an unencrypted form.
A message transfer agent (MTA) is used to relay emails from one mail server to another. MTA uses SMTP and deals with receiving messages and relaying messages. It is important to note that MTA-s do not have to communicate with each other over TLS/SSL. If one of the MTA-s does not support TLS/SSL then the email is exchanged in plaintext. In case an email is sent to a mail server that does not support TLS/SSL then it is guaranteed that the email will be transmitted in plaintext at least in one transmission step. Therefore, STARTTLS and secure mail exchange protocols do not provide confidentiality during the transmission of the email as one of the mail servers might not support TLS/SSL. Even when all links during the transmission of the email are secured by encryption the contents of the email can leak as the mail servers are able to read the email in plaintext. Therefore, one needs to trust the MTA-s while sending emails but this is not a very wise thing to do when the contents of the emails are confidential.
In addition, it is important to notice that the metadata in the header of the email can not be encrypted as this information is required for relaying the emails. Therefore, it is possible to intercept the metadata that contains: sender email address, recipient email address, sender ip, recipient ip, date, time, title of the email. This data is public and the mail exchange protocols do not provide means to protect such information.
Risks that are related with email communication:
- the contents of the emails can be intercepted and saved
- the contents of the emails can be modified
- the contents of the emails can become public information
- it might be possible to use someone else's identify
A very good overview of the security issues of email is given by the article Neither Snow Nor Rain Nor MITM ... An Empirical Analysis of Email Delivery Security (2015). It is written by
Solution for the privacy and security issues
There is no ideal solution for the described security and privacy issues. The best available solutions use end-to-end encryption but they are not widely used. In end-to-end encryption the message is encrypted in the sender's computer and decrypted in the recipient's computer, which means that the intercepting party can not get access to the contents of the message. End-to-end encryption of emails is provided by PGP/GPG and S/MIME. As an alternative, it is possible to use other communication means that provide end-to-end encryption. One popular alternative used to be Off-the-Record Messaging (OTR) but nowadays Signal is considered to be one of the most secure communication applications.
Today we are going to use PGP/GPG to encrypt and sign emails. You will also have to use Signal in order to solve one of the homework tasks.
S/MIME
S/MIME (Secure/Multipurpose Internet Mail Extensions) is a mail exchange standard that is based on public key infrastructure. In addition to providing confidentiality by using encryption, non-repudiation by using signing, it also provides authentication and message integrity. Similarly to HTTPS, each end user should have a key pair and a certificate that binds the identity (email address) with the corresponding public key. The certificate has to be signed by a trusted certificate authority (CA).
In order to encrypt emails with S/MIME one would need to have the public key of the email recipient. Usually the public key can be accessed if the recipient has previously sent a signed email which contains the certificate and therefore also the public key.
S/MIME is supported by several mail clients but it is rarely used as one would have to get a certificate that is signed by a CA. However, S/MIME is widely used in Estonia as each ID-card owner has a certificate signed by a CA. It is possible to send encrypted and signed emails using the Estonian ID-card. In order to do that one would have to search for the corresponding certificate and then use the email address that is in that certificate. The email addresses in the ID-card certificates are of the following format: Surname.Lastname[.X]@eesti.ee.
PGP and GPG
Pretty Good Privacy (PGP) is a computer program that was created in 1991. It allows to encrypt and sign documents and emails. Similarly to S/MIME PGP is based on public key cryptography. Each PGP user has a key pair but it differs from S/MIME by the way how keys are managed. PGP is not centralized, i.e., it does not use public key infrastructure. Therefore, one does not have to get a CA to sign a certificate which makes it easier to start using PGP.
Instead of using public key infrastructure where trust relationships are created between CA-s, PGP uses something that is called a web of trust. In the web of trust everyone can choose how much trust is given out and then the decision is bound by signing the corresponding certificate. If many people enforce the trust by signing other peoples certificates then a social network of trust is created where trust is a transitive relation as shown on the following figure.
PGP trust relations. An ordinary arrow denotes that a certificate has been signed and the dotted arrow denotes a trust relationship. For example, as Dave trusts Alice and Alice has signed the certificate of Bob, then by the transitive relation Dave also trusts Bob, i.e., Dave believes that the corresponding public key really belongs to Bob. (Source)
Key servers are used to manage the trust relationships. One can download a certificate from the key server and also upload new signed certificates to the key server. Therefore, everyone can create new trust relations and publish them in the key servers.
Actually, PGP is an old commercial software product and therefore it is not used anymore. A standard called OpenPGP was created based on PGP in order to make the functionalities provided by PGP widespread. OpenPGP is supported by different software products which makes them interoperable. The most common and widespread software that follows the OpenPGP standard is GNU Privacy Guard (GnuPG or GPG). GPG is an open-source program that is available for free.
GNU Privacy Guard (GPG)
GPG allows to exchange emails and attached files in a secure manner. GPG allows to encrypt and sign emails which means that a third party is not able to secretly modify or read the contents of the emails. In addition, when an email is signed the the receiver can be sure who sent the email (actually, the receiver only knows that the sender had access to the corresponding private key).
GPG is considered to be one of the safest methods for encrypting data. However, GPG is not very popular and only a small group of people communicate by using GPG. Therefore, one can usually start using GPG only if the receiving party is instructed on how to setup GPG. There might be a situation when someone, e.g. a client, wants to submit confidential information and GPG provides a solution for this problem. Estonian citizens can use the encryption and signing functionalities provided by the ID-card but if the client is a foreigner then ID-card may not be an option. In such cases one could create an OpenPGP key pair for himself / herself or for the company. However, just creating the key pair is not enough, the key pair would also have to be published so that the clients could find the key. One option is to publish the key in addition to the key servers also to the company website in case the website is configured to use HTTPS.
Tasks for the lab
You should be using a virtual machine if you are attending the lab. The virtual machine already contains most of the required software. If you already have a virtual machine (you were sitting in front of the same computer last time) then try to start it. If you don't have a virtual machine, then get a new one.
Instructions:
- Open "Oracle VirtualBox", you can find it from the start menu.
- Open from the menu "File -> Preferences -> General -> Default Machine Folder" and select the folder to be "O:\infsec\" or something similar if that folder does not exist. Click on the yellow folder icon in order to browse the existing folder. When you are done click OK.
- Import the virtual machine: "File -> Import Appliance ... -> click on the yellow folder and then browse into O:\infsec\information_security_2018.ova. Click "Open" and then "Next".
- Change the name of the virtual machine by modifying the field "Name". To do that double click on row next to "Name". Now click "Import" and wait a few minutes until importing has been finished.
- Once importing has finished then you can start the virtual machine. To do that click on the name of the virtual machine and then click on the greed start button. Alternatively you can just double click on the name of the virtual machine.
- The hard drive of the virtual machine is encrypted with VeraCrypt. Enter the password that is written on the whiteboard and then hit enter twice (pim is empty). You can use the same password to log into the user account in Windows.
- Now skip the following two subsections and start to read the subsection about Mozilla Thunderbird and GPG. You already have Thunderbird, Enigmail and GPG installed in your virtual machine.
Windows: Gpg4win and the Thunderbird extension Enigmail ( skip this if you are in lab )
GPG4win logo (Source)
The GPG version on Windows is called Gpg4win and it contains the following tools:
- GnuPG - the main tool that allows to encrypt / decrypt / sign / verify
- Kleopatra - certificate manager
- GPA - an alternative certificate manager
- GpgOL - a plugin for Microsoft Outlook that allows to encrypt emails
- GpgEX - a plugin for Microsoft Explorer that allows to encrypt files
- Claws Mail - a mail client that supports GPG
- Gpg4win Compendium - documentation in english and in german
How to install Gpg4win to your computer ( skip this if you are in lab ):'''
- Navigate to the website http://www.gpg4win.org/
- Click on the button "Download Gpg4win" and choose the newest version of the software
- Download the installation file and then open it
- Click "Yes", "Next"
- Choose to install all components and click "Next"
- Click "Install", "Next" and "Finish"
Tasks regarding Mozilla Thunderbird & GPG
Mozilla Thunderbird is a popular email client that can be downloaded from here: Download Thunderbird in your language. In order to send encrypted and signed emails using Thunderbird one would have to install the extension Enigmail, which adds support for GPG.
Task 1: install Mozilla Thunderbird (skip this task if you are participating in the lab )
- Navigate to https://www.mozilla.org/en-US/thunderbird/all/ and choose the suitable language
- Download the installer for your operating system (Windows for the virtual machine)
- Open the installer file and use the default options to install Thunderbird: Yes -> Next -> Next -> Install -> Finish
Task 2: install Enigmail (skip this task if you are participating in the lab )
- Open Thunderbird
- Check if you already have Enigmail menu in Thunderbird. If you have then go to task 3.
- Navigate to "Tools" -> "Add-ons" -> "Search all add-ons"
- Search for "Enigmail"
- If found then click "Install"
- Click "Restart now" to activate the Enigmail
Task 3: link your email account with Thunderbird
- Link your email account with Thunderbird as then you can create a keypair for your email aadress. When choosing the email account make sure that it does not forward emails as otherwise you can't complete the lab tasks. In addition, two factor authentication might be a problem unless you are using Gmail (Thunderbird supports 2FA with Gmail). If you do not have a suitable email account then create a new email account for this lab (this might take some time). We have found out during the last years that the following accounts are not suitable: @mail.ee, @hot.ee, @mail.com, @yahoo.com as they consider encrypted emails as spam or block them.'''
The main condition is that you will have to be able to send and download both regular and encrypted emails with Thunderbird. You can create a new free email account using Gmail, AOL or Outlooki. - Link the new email account with Thunderbird
- Select "File -> New -> Existing Mail Account..."
- Fill in the forms by using the email and password of the new account and click "Continue""
- Hopefully mail servers are detected and you can click "Done"
- Lets prevent the time consuming synchronization of old emails
- Right click on your email account and select "Settings"
- Synchronization & storage -> Message Synchronizing -> remove tick
- Synchronization -> Disk space -> Select: "Synchronize the most recent 1 Days".
- Select "OK" and then restart Thunderbird (close and open)
Task 4: create a new key pair for the linked email account
- Navigate to: "Enigmail" -> "Key management"
- In the OpenPGP Key management window choose "Display All Keys by Default"
- Create a new key pair: "Generate" -> "New key pair"
- Choose the linked email account
- Enter a passphrase that will protect your secret key (remember the key)
- In the practice session you should set your key expiry to 1 month
- Select to generate either 3072 bit RSA key or ECC key
- Click "Generate key"
- Move your mouse and hit buttons on the keyboard to generate randomness that is used for generating the new keypair
- Once complete click on "Generate Certificate" that creates a revocation certificate
- Save the revocation certificate on the disk (this can be used to invalidate the public key)
- The revocation certificate can also be generated later by clicking "Enigmail" -> "Key management" -> "Generate" -> "Revocation certificate". It is important that you would have a revocation certificate if you start to use GPG.
- We have to change the configuration of Enigmail before we can move to the next step. Go to the Enigmail menu an choose Preferences -> The Display / Hide Expert Settings and Menus -> Keyserver. Remove the names of keyservers from the "Specify your keyserver(s)" field and enter the keyserver that is written on the whiteboard. The name of the keyserver is also written in the homework task. Now click ok and move to the next step.
- Now you have a key pair but it is not very useful if no one knows your public key. Therefore, as a next step we will upload the public key to our key server (the address of the server is written on the whiteboard). If you would like to publish your public key for your real email address then you should pick one of the most common key servers that are also suggested by Enigmail.
- Navigate to "Enigmail" -> "Key management" -> "Keyserver" -> "Upload Public Keys"
- Click OK
Task 5: find the public keys of your left and right neighbours
- You will need to know the public keys of your contacts in order to send them encrypted messages. The public key is also required to verify the signatures that are created with their private keys. To search a public key:
- Navigate to "Enigmail" -> "Key management" -> "Keyserver" -> "Search for keys".
- Enter either the name of the person, the email of the person or the ID of the public key
- Enter the key server from which the key will be searched from
- Now find the public keys of your left and right neighbours
Task 6: send a signed email to your left and right neighbour
- To send an encrypted email the mail account has to be linked with the corresponding key pair but we have already done this step. Now when you write an email you can choose from the Enigmail menu to switch on signing or to switch on encryption. This can also be done with the corresponding icons in the toolbar.
Task 7: try to verify the signed email from your left and right neighbours
Task 8: send an encrypted email to your left and right neighbours
- To send an encrypted email you will need to know the public keys of your neighbours. Create a new email and then pick from the Enigmail menu to switch on encryption.
Task 9: Find the public key of the supervisor (the supervisor will write the correct email address and key ID on the whiteboard) and then send him an encrypted and signed email. This email should contain your name. This is one part of the homework - if you manage to do this in the lab then you have already done a part of the homework.
- Send your lab supervisor an encrypted and signed e-mail using PGP/GPG. It should be a single mail message that contains your name, signed and encrypted at the same time. (3p)
- Upload your public key to the key server
infsec.cs.ut.ee - To encrypt, use these public keys: Kristjan Krips (5BFC8B9D), Toomas (3D90 CCB5).
- Be sure to include some text in the mail message itself, otherwise there is nothing to sign or encrypt.
- Upload your public key to the key server
Task 10: If you were using a virtual machine in the computer class then remove the linked email account from Thunderbird.
Task 11: Use Signal to send end-to-end encrypted messages. This is part of the second homework. If you are not able to try out Signal then send an email to krips ät ut ee and he will give you a replacement task.
Signal is an open source messaging application that uses end-to-end encryption. It works on Android since version 4.0 and on iOS since verion 9.0. Once Signal is activated on your phone you can also use a desktop client (it is supported on Windows, MacOS and on Debian based Linux distributions). You can get helpful information from Signal Messenger Basics and Signal Security.
- install Signal from Google Play / App Store
- Signal uses contact list in order to find contacts. Therefore, you can see if anyone from your contact list is using Signal. We are using a prepaid card for the homework task. Add temporarily to your contacts the following number: five eight six seven one four five nine.
- Now you are able to send messages to the test account. Of course if you have others in your contact list then you could contact them also using end-to-end encrypted messages.
- To claim one homework point send a self destructing message to the test account. The message should contain your name. If you are not able to install Signal or do not want to do this task then ask for a replacement task from krips ät ut ee.
Task 12 (voluntary): In case you have spare time and want to test out other tools that support pgp then visit: Mailvelope. This tool is recommended by the New York Times.
What should be done differently when creating a key pair for your own email account
- the secret key should be protected by a secure password
- the expiry date of the key should be longer but do not set it to never expire
- the secret key has to be kept safe to avoid leaks and to be sure that you will be able to decrypt messages in the future
- you should create a revocation certificate which allows to mark a public key invalid - it is not possible to delete an old public key from the key server, you can only mark it to be invalid by using the revocation certificate
- you should publish your public key to a well known key server
- in addition you may connect your public key with your identity in the Internet, e.g., with your Twitter account, with your homepage, etc.
Mac OS X: GPGTools
GPG also works on Macs, on OS X it is provided by GPGTools. GPG Suite comes with GPG for Mail, GPG Keychain, GPG Services and MacGPG. If you would like to use GPGTools, then navigate to their website and click on "Download GPG Suite".
After installation, open GPG Keychain Access and create a new keypair for you email account. If you already have a keypair for the account then it is possible to import it. As a next step create a revocation certificate, to do that navigate to "Key" -> "Generate Revoke Certificate". Under the same menu is an option that allows to publish the public key and also to search for public keys of other people.
Now it should be possible to send encrypted and signed emails using Mail. In order to send an encrypted email one would have to start writing the email and then click on the encryption button, the same holds for signing. It is important to note that one can send encrypted emails only to these contacts whose public key is available in the GPG Keychain Access. I.e., before sending an encrypted email one will have to search for the corresponding public keys. It is very likely that the contacts do not have an OpenPGP public key and therefore it might be necessary to instruct them on how to use GPG.
Problems related with OpenPGP and GPG
An overview of the problems related with PGP is given by Matthew Green: What's the matter with PGP?.
- few users and it is not trivial to use PGP
- difficult to create trust relationships
- email metadata is not encrypted
- it is difficult to share and verify the long public key
- queries to the key-servers are not encrypted
- no support for perfect forward secrecy
- access to the password that unlocks the private key is constantly required
- secret key is kept in RAM even when it is not used
- people do not know what is PGP or how to verify emails signed by PGP (this problem is illustrated with the following comic)
Secure email exchange in a browser
PGP and S/MIME can be used only in supported mail client software (Outlook, Thunderbird, Mail.app, etc.) but many users only use the online version of mailboxes like Gmail, Outlook.com, Yahoo! mail. How could such users increase their privacy and security? They can not use S/MIME or PGP as the web browser can not access the secret key without a special browser extension and it is not possible (also it would not be wise) to store the secret key in the service providers server. In addition, using S/MIME or PGP in the web browser would probably mean that the cryptographic operations would have to be implemented in JavaScript, which would mean that the service provider would have to be trusted.
In 2014 both Google and Yahoo! announced that they are developing a browser extension that would allow to encrypt and sign emails when using their online mailboxes. The browser extension developed by Google is called End-to-End and it is based on PGP. The browser extension developed by Yahoo! used the same code base.
OpenSSL - voluntary material & voluntary tasks
OpenSSL is a popular open source cryptograhy library that contains an implementation of TLS. In order to see how this library is used you may want to try out some simple commands. However, OpenSSL does not come bundled with Windows and therefore, we will first have to get access to a suitable platform. The simplest way for you is to log in to the university server which runs on Linux and has OpenSSL installed. For that you will need a program called PuTTy.
- Download PuTTy to your computer and try to verify the installer
- Download the latest development snapshot of putty for Windows from: http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html
- In addition, you can also download the signature file that corresponds to the executable that we just downloaded. It is important that the signature would be saved to the same directory. Then it is possible to verify that the binary is correctly signed.
- If you are a bit paranoid and want to verify the signature of the PuTTY executable then follow the instructions. You can use gpg to verify the executable. However, this requires that gpg is already installed in your computer.
- Open cmd.exe by going to "Start" -> "Search Programs and Files" and then write "cmd" and hit Enter button.
- Navigate to the folder where the three downloaded files are located. You can use the command "dir" to see the contents of the current directory, the command "cd directoryName" to change directory and the command "cd .." to move one level up in the directory tree.
- Import the public key by entering:
gpg --import <add real name of putty's public key file here> - Verify the installer by entering:
gpg putty-<add real date here>-installer.exe.gpg putty-<add real date here>-installer.exe - You should see the following message: Good signature from "Putty Development Snapshots... Now you should be more or less certain that the software is authentic and thus you can start the installation.
- Install PuTTy
- Make sure that a shortcut will be created on the Desktop.
- Use ssh to log in to the university server
- Start PuTTy
- Write "math.ut.ee" to the Host name (or IP address) field
- Click "Open" and click "Yes" to accept the servers public key
- Enter your university username (same as in the Study Information System)
- Enter your university password (same as in the Study Information System)
- Check that you are logged in: you should see yourusername@math: on the terminal
- Lets try out the following OpenSSL commands:
- openssl version
- openssl speed aes-128-cbc
- openssl speed rsa2048
- openssl speed sha256
- openssl genrsa -out key.pem -aes256 4096
Further reading
- Neither Snow Nor Rain Nor MITM ... An Empirical Analysis of Email Delivery Security
- Encrypted e-mail: How much annoyance will you tolerate to keep the NSA away?
- Guide for operational security while using PGP
- Matthew Green: What's the matter with PGP?
- PGP
- GPG, Thunderbird, Enigmail
- PGP in a web browser