1. Overview of lab 2 :
Welcome to the 2nd lab. Here is a short action list that we will do during this lab:
- Accessing ETAIS
- Creating a public/private key pair
- Creating a new CentOS 8 virtual machine in Cloud
- Configuring VPN Client
- Accessing Server over SSH
- Adding
scoring
account and its public key with no password for sudo commands - Update dnf configuration
- Following validations have to be passed for this lab to be completed:
- Your machine must ping from the scoring server.
- Scoring user must be able to log into your VM using our key.
- Scoring user must have passwordless sudo access.
- Scoring user must not have a password set.
2. How to login to ETAIS:
For the rest of the labs, we use a cloud service called ETAIS. ETAIS’s main aim is to serve Estonian students and researchers. For access, you have been sent an email with instructions on how to log in. You have also been invited into the System Administration
organization, and have a private project in that organization.
- Log into ETAIS system
- Go to the page: https://minu.etais.ee
- And login by clicking
Sign in with MyAccessID
- After that search for your institution (University of Tartu) and enter your (UT) username and personal password.
- PS! If you prefer other options might work also ID card, Mobile ID, or another institution's username and password, and so on, but it is up to you to get them working.
- Look for a button called "Select workspace" (up and center)
- Select workspace and your project
- If not available please go to the personal e-mail you provided to University as the primary contact e-mail and look for an e-mail from
selfservice@etais.ee
and click on a linkthis page
in it.
- If not available please go to the personal e-mail you provided to University as the primary contact e-mail and look for an e-mail from
- If still having problems please contact your lab instructor or Sander Kuusemets
sander.kuusemets@ut.ee
If you did successfully login, you should be able to see the ETAIS Dashboard
as follows:
3. Creating a public/private key pair
Before you continue we need to set up public / private key pair so you could use them to connect to the machine.
Public key cryptography, or asymmetrical cryptography, is any cryptographic system that uses pairs of keys: public keys which may be disseminated widely, and private keys which are known only to the owner. This accomplishes two functions: authentication, where the public key verifies a holder of the paired private key sent the message, and encryption, where only the paired private key holder can decrypt the message encrypted with the public key.
In a public-key encryption system, any person can encrypt a message using the receiver's public key. That encrypted message can only be decrypted with the receiver's private key. To be practical, the generation of public and private key pairs must be computationally economical. https://en.wikipedia.org/wiki/Public-key_cryptography
Make sure you do not lose this key, once Your VM has been created the only way to change the key is from inside the VM. If you lose your copy and are locked out of the VM, you will have to create a new virtual machine!
Generating key pair and uploading the public key to the SSH server
To be done on personal PC/Laptop and
- In Linux, macOS, and Linux subsystem for Windows ( PuTTy manual later):
- Open command line
- Use
ssh-keygen
command to generate the 4096 bit long RSA key (try to figure out the proper command options you want to use. If asked where to store the key use the default location of:${HOME}/.ssh/id_rsa
- If asked about a password you may consider protecting the private key using a passphrase for stronger security, but it is also OK to leave it empty (in this case the private key has to be kept really secret).
- Now two files should be added into
.ssh/
directory of your home folder:id_rsa
which is a private key and should be kept secret (by the way what are the proper permissions assigned to the file ?)id_rsa.pub
which is a public key and may be shared (by the way what are the proper permissions assigned to file ?)
- Once the key pair is generated we may set up the SSH server to use public-key authentication:
- Copy public-key to memory and then continue with Manual on where to paste it in ETAIS:
$ cat ~/.ssh/id_rsa.pub
- Copy public-key to memory and then continue with Manual on where to paste it in ETAIS:
- In Windows:
Use Puttygen application to generate the key pair:
- Download Puttygen
- Set the password for the private key and save it to your hard drive.
- Take the puttygen (public key) output and copy it to the
ETAIS
(next part in manual).
NB! Do not save the public key into a file and copy it to the server, it will not work, it has a different format.
- Puttygen illustration: https://www.youtube.com/watch?v=-92wEg68SKQ, https://www.ssh.com/ssh/putty/windows/puttygen
- Copy the public key to ETAIS.
- Log Into ETAIS
- On the left menu (under your name) Click
Details
->SSH keys
- Right side of the screen look for
Add key
and click on it - Add a name to your SSH key that should contain your
Last name
Paste your public key
and clickAdd key
Once the public key is successfully added, you should be able to see it in the list of SSH Keys
in the left-side menu in ETAIS:
4. Creating a new Virtual Machine
- Log Into ETAIS
- From the top choose your Workspace
System Administration
->Your personal project
->Select
- On left menu go to
resources
->VMs
->Add Resource
->Virtual Machine in <your project name>
- VM name =
Your university study book number (matriklinumber)
- Image =
LTAT.06.003
- Flavor =
m1.medium
- System Volume Size = 20 GB
- System Volume Type =
prod2 (Dedicated production HDD)
- Data Volume Size = 1GB
- Data Volume Type =
prod2 (Dedicated production HDD)
- SSH public key =
Your public key that you added before
- Security Groups = add
ssh
andping
(can be done by clicking in white area afterdefault
) - Networks =
Auto-assign Floating IP
- Description =
Add your Full name, matrix number and course name (System Administration 2022)
- VM name =
- Check everything twice and press
Add to cart
, and go through with the purchase.
Example VM configuration will look as follows: (WARNING: the Provider value will be specific to your project, so it will be different!)
Once the VM is successfully created it should be visible in the list of VMs in ETAIS (''Resources -> VMs in the left-most menu)
General Network Topology
You may notice a VM has Internal and External IP listed. Each project suggests a private network of 192.168.42.0/24, this is your project's private LAN, an IP from this network will be assigned to your VM's eth0 network interface and also listed in the Internal IP column (VM list in ETAIS). Project private LAN is connected to the University's Campus network (172.17.0.0) through a router device that is specific to your project (each student will get a 192.168.42.0/24 private network but this is not the same one!). Router devices are created automatically once the project is created and the network provider is chosen. The only possible provider for our course cloud is UT Campus (172.17.0.0). Therefore router devices will get the corresponding External IP address assigned, resulting in a unique address from the 172.17.0.0 subnet. The Internal address of the router is always set 192.168.42.1 and this becomes a default gateway for our VM. As we only create 1 VM in our private network (192.168.42.0/24) the router applies the Source/Destination NAT (network address translation) between the IP address of a VM (192.168.42.a) and the External address of a Router (172.17.x.x). As a result, all packets arriving on 172.17.x.x are sent further to 192.168.42.a (called destination NAT). The opposite way around (source NAT): all packets that are passing through the router (sent from VM into the world) are getting the source address rewritten (192.168.42.a -> 172.17.x.x). This is a default setting for any home/small-office router. If both are applied the Router becomes transparent for a VM: VM becomes accessible behind the Router's External address. This is why the Router's External IP is enlisted as VM's External IP (right next to VM's Internal IP).
Network topology diagram
Question: What will happen if there is 2 VMs instead of 1 inside a private network 192.168.42.0/24
5. Configuring the VPN Client
To access your newly created Virtual machine you need to be inside of University of Tartu network. In the classroom or inside the University of Tartu eduroam
network it is possible to access your machine without a VPN, but we do recommend setting up a VPN for remote access outside the University network. You can skip this step if you have already configured a UT VPN.
A virtual private network (VPN) extends a private network across a public network and enables users to send and receive data across shared or public networks as if their computing devices were directly connected to the private network. Applications running across the VPN may therefore benefit from the functionality, security, and management of the private network. VPNs may allow employees to securely access a corporate intranet while located outside the office. They are used to securely connect geographically separated offices of an organization, creating one cohesive network. ... https://en.wikipedia.org/wiki/Virtual_private_network
- Please refer to the University of Tartu official manual of how to set up VPN in your personal machine if you have not done this already:
Once the VPN connection is established, check the connection with your newly created Virtual machine. Use ping command with the External IP of your VM
$ ping 172.17.64.250
6. Access machine over SSH
Secure Shell (SSH) is a cryptographic network protocol for operating network services securely over a network. The best-known example application is for remote login to computer systems by users.
SSH provides a secure channel over an unsecured network in a client-server architecture, connecting an SSH client application with an SSH server. Common applications include remote command-line login and remote command execution, but any network service can be secured with SSH. https://en.wikipedia.org/wiki/Secure_Shell
Connecting to Cloud resources (VM-s) over SSH is preferred. Therefore SSH is installed in your system by default, as the cloud infrastructure takes care of bootstrapping and installing your CentOS 8 OS. The other option is to use the slow network console from minu.etais.ee, but usually this is reserved for emergency purposes.
Accessing remote shell of the VM over SSH
Command line utilities
ssh (Linux,MacOS, WSL)
In macOS and Linux hosts the ssh
client is by default installed and accessible in the command line. Starting with Windows 10, a Linux subsystem was created that runs a pseudo-VM in Windows, which also has ssh
installed by default. To access the remote shell of the example.com host and user test the following command should be provided:
$ ssh test@example.com
Try accessing the remote shell of your VM using the Floating (External) IP address of your machine. Your username (default: centos
) but you do not need a password since we are using key-based authentication. If all is good, your login should work, if you generated a new key or the key does not authenticate properly, you must re-create your VM with the correct keys, as the key is modified only during VM creation.
ssh <user>@172.17.X.X
- The SSH server should not ask for password anymore in case you did not provide additional pass phrase on your private key
id_rsa
. - If needed use
-i
parameter to specify key file.
Desktop Applications
Putty (Windows)
**Download Putty
Putty is a portable executable, so just open Putty.exe
.
Testing the public key authentication
First of all, make sure you know the IP of your newly created VM. It will be something like 172.17.XXX.XXX
Instead of username and password let's use the key we generated earlier. Now you should be able to login to your VM over SSH using the private key:
- In Windows
- Use the Putty application and set up a new saved session:
- On the putty startup screen write your External IP address (172.17.64.X) into to
host name
field - this will be the session name (so you can reuse it later). - Click on the
Connection
(in the left vertical panel) then chooseData
.- Write the
login
of your normal user (the default one iscentos
).
- Write the
- On the putty startup screen write your External IP address (172.17.64.X) into to
- Under
Connection
->SSH
->AUTH
, browse for theprivate key
you generated with thePuttygen
. - Go back to the
Session
submenu, add aSA-172.17.64.X
name inSaved Sessions
box and clickSave
.
- Use the Putty application and set up a new saved session:
- Now the saved session can be used:
- Use the Putty application:
- On the putty startup screen, in the
Saved Sessions
box select your saved session, then clickLoad
- Click
Open
on the main window.
- On the putty startup screen, in the
- The SSH server should not ask for a password anymore in case you did not provide an additional passphrase on your private key.
- Use the Putty application:
7. Adding root password
VMs in ETAIS by default ship without passwords. Meaning you can only gain access to the VM with the correct public keypair. In this course, the root
user should have a password, as it allows for disaster recovery in case something happens to your key/VM through the web console. In general system administration, root
passwords are usually not used, or only for disaster recovery purposes. Having a password with full control of your services/server is a security risk.
As a root
user set the password for the root
account
# passwd root
You will be prompted to enter the password twice, don't be alarmed when you can't see your password being typed on the screen. It's a security feature of Linux as to not give away the length of your password. Also make sure you remember or write down the password, in case you need it.
Go to ETAIS next to your VM's name choose Actions and from the drop-down menu choose Open console
. This will cause a pop-up to open a new tab. You might need to allow pop-ups for ETAIS in your browser (you will be prompted to do so). Click inside the black console box to force it into focus and try to log in with user root
and the password you just set.
8.Adding scoring account
Now we will add a scoring user which will be used by our lab instructors and an automatic scoring service to log in and check your machine.
- Add new user called
scoring
(DO NOT give it a password) $ sudo adduser scoring
Now let's give our lab instructors and automated scoring system access to your VM over SSH using a scoring account. The public key for the scoring user is available for download here, and it works exactly like your own public-private keypair for ssh https://scoring.sa.cs.ut.ee/files/id_rsa.pub.
- Now in your VM please log in to
scoring
account you have just created, or just switch to it usingsu - scoring
command. (If you get a permission denied error, think a bit - you need elevated permissions to log to a new user without confirming a password(which we don't have)!) - Now download the scoring server's public key in your VM, use the
wget
(you might need to install it) command for that: scoring@host:~$ wget https://scoring.sa.cs.ut.ee/files/id_rsa.pub
- Alternatively, you can put the URL into your browser and copy it from there.
If the file was downloaded properly you can check the content of the file
scoring@host:~$ cat id_rsa.pub
Now let's add the downloaded public key to the list of authorized keys for scoring account:
- If directory
.ssh
does not exist in scoring's home, please create it.scoring@host:~$ mkdir .ssh
- Make sure the directory is owned by scoring user and only it can read-write-execute in it
# chown scoring:scoring .ssh
and# chmod 700 .ssh
- Copy the contents of
id_rsa.pub
into.ssh/authorized_keys
scoring@host:~$ cat id_rsa.pub >> .ssh/authorized_keys
- Make sure the files are in appropriate permissions
.ssh
folder in Read-Write-Execute only permissions forscoring
user only (no permissions for others)authorized_keys
in Read-Write permissions only for thescoring
user
What we did was add the scoring server's public key into a special file named authorized_keys
, the contents of which are checked by the SSH service in the VM for each login to assess authentication methods for that particular user. This process is identical to the VM setup scripts adding *your* key under the authorized_keys
file under the centos
user(you are encouraged to check the contents of /home/centos/.ssh/authorized_keys
). The format of the file allows for multiple keys as well, one per line. When manually adding new keys to any user, make sure the formatting is correct and that the key header ssh-rsa
is attached to the key.
If everything was done properly the remote scoring can now log in using its private key. You, unfortunately, cannot test it manually, but you should see that in a few minutes from the scoring board.
The last thing to do is to give the scoring user rights to execute commands in root permissions.
We need the scoring user to be able to run programs that need root access, but we are not going to give root privileges to the scoring backend (root password is unknown and root user cannot log in over SSH). To give a regular user the ability to execute a particular command in superuser rights there is a tool called sudo
. sudo
allows running programs with the security privileges of another user, which, by default is the root user.
For more information on sudo read a short review on Debian wiki.
Check that the package sudo
is installed
- Remember what command we were using for installing packages?
Rely on sudo documentation and make changes so that the scoring user can:
- Execute sudo command without asking for password
centos@mohan-test-vm:~$ sudo visudo
and add
scoring ALL=(ALL:ALL) NOPASSWD: ALL # under where users like root are specified
- Depending on your editor, you need to know how to save.
- Default is
vim
.
Take care in not making syntax errors, as this can break system functionality.
Ideally, the following should just list files without any additional prompts:
scoring@host:~$ sudo ls -l /root
Note the user under which the command was run
9. Perform backups
Whenever you are about to introduce some portion of changes to your system it might be wise to back up the current state first. Configuration mistakes might result in locking the network access. Once network access is lost - you can't control your VM over SSH anymore, and it might be useful to restore from backup (not always). There is a recovery console (Action
button in ETAIS next to your VM has Open console
option), but that only works if you have set a root
user password.
ETAIS infrastructure does allow you to perform the snapshots of your VM image. By creating a snapshot you are saving the state of the whole hard drive, and once the snapshot is created you can proceed with dangerous actions knowing that you can always recover to the last snapshot's state in case you misconfigure something. But this should not happen often, and keeping multiple snapshots is not necessary. Another way of backing your VM state is to automate VM creating with Ansible, something we will be covering in the next lab. Ansible scripts allow you to automate the steps you do in this course, and in the case of VM corruption, easily re-setup your machine to the last state. Using Ansible is also encouraged during the exam and a properly configured Ansible can automate the whole exam, allowing easy pass of the course sometimes in minutes.
In ETAIS interface issue the snapshot of the hard-drive
- ETAIS -> Resources -> VMs -> Your VM Name -> VM snapshots -> Create
Once you have checked that the snapshot creation worked, feel free to delete it. This is only a tool we introduce for you, but using it is not mandatory.
10. Security groups.
From this lab onward we need to sometimes open ports for our VMs, to allow config of the different services we set up. Some of these security groups should be preconfigured in the course project already, but configuring the firewall of your VM is entirely up to You. Below is a small guide on how to configure firewall rules if the guides ask for them.
- Go to ETAIS -> Resources -> VMs -> Your VM -> Manage security groups -> Create -> Name: <something descriptive> (Description: <description>) -> Submit.
- For the newly created security group look at -> Actions -> Set rules -> Add rule -> Ethernet type: IPv4 -> Direction: Ingress -> IP protocol: TCP -> From port: 1000 -> To port: 1000 -> Remote CIDR: 0.0.0.0/0 -> Remote security group: None
- Press Submit.
We have created a dummy security group for TPC port 1000. One last step is to actually attach it to the VM, as creating a group doesn't apply immediately to all VMS (Why? Most projects have more than one VM, you don't want to apply wide rules to all of your VMS).
- Go to ETAIS -> Resources -> VMs -> Actions -> Update Security groups -> Add the newly created Security Group DNS to the list of Security Groups -> Submit
When you open any other port, you also have to do it in the ETAIS interface. This is not a one-time thing and has to be done for each port/protocol/service we install into our machines. One security group can also have multiple ports/protocols in them.
11. Updating DNF:
DNF (Dandified YUM) is a new package manager for the CentOS platform. This can be used to install, update, remove software and handle dependencies.
Get updates and install them:
# sudo dnf update
- Use DNF to install your favorite text editor and the package named
sl
.
That is everything for this lab, you should now have all the tools to start completing the following labs.