DevOps: tarkvara tarnimise ja käituse automatiseerimine 2021/22 sügis

Practise Session 2 : Working with Docker Container Engine

Make sure that you have already gone through Lab-01

Docker is an open-source platform for developing, shipping, and running applications. Docker enables you to separate applications from infrastructure and deliver your applications quickly from sandbox to production environments. The DevOps is primarily used to overcome 'Dev' and 'Ops' problems, and Docker seems to solve most of them by providing seamless control overall inevitable changes in development, production, and staging environments. So, the aim of an experiment is to get acquainted with docker fundamentals to advanced features such as to build and push the container images.

Introduction to Docker

Docker containers allow the packaging of your application (and everything that you need to run it) in a “container image”. Inside a container, you can include a base operating system, libraries, files and folders, environment variables, volume mount-points, and your application binaries.

• Docker image
  • Lightweight, standalone, executable package that includes everything needed to run a piece of software
  • Includes code, a runtime, library, environment variables, and config files
• Docker container
  • Runtime instance of an image - what the image becomes in memory when actually executed.
  • Completely isolated from the host environment.

Build, Ship, and Run Any App, Anywhere

The below Figure shows the architecture of docker installation in the ETAIS cloud and its interaction with the Docker hub.

• Docker is available in two editions: Community Edition (CE) and Enterprise Edition (EE)
• Supported Platform: macOS, Microsoft Windows 10, CentOS, Debian, Fedora, RHEL, Ubuntu, and more on https://store.docker.com/.

References

Referred documents and websites contain supportive information for the practice.

Manuals

1. Docker fundamentals
2. Docker CLI
3. Building docker image
4. Docker swarm
5. Docker architecture

Prerequisites

• Make sure that you have already gone through Lab-01
• Create a login account in docker hub using this link and sign up.

Exercise 1. Installation of docker in an ETAIS Virtual Machine

In this task, you are going to install a docker in ubuntu OS and try to run the basic commands to make you comfortable with docker commands used in the next tasks.

• Create a virtual machine (image = ubuntu18.04 OS, flavor = m3.xsmall) as carried out in Lab-01 and connect to the virtual machine remotely using gitbash or any other SSH client.
  • NB! Extra modifications required to change docker network settings:
    • Create a directory in the virtual machine in the path: sudo mkdir /etc/docker
    • Create a file in the docker directory: sudo vi /etc/docker/daemon.json
    • Copy the following script:

{
"default-address-pools": [{"base":"172.80.0.0/16","size":24}]
}

• Update the apt repo sudo apt-get update
• Install packages to allow apt to use a repository over HTTPS:

sudo apt-get install apt-transport-https ca-certificates curl gnupg-agent software-properties-common

• Add Docker’s official GPG key

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

• Use the following command to set up the stable repository.

sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"

• Update the apt package index.

sudo apt-get update

• Install the docker

sudo apt-get install docker-ce docker-ce-cli containerd.io

NB! To run docker commands with non-root privileges

• Create a docker group (If it's already created then ignore): sudo groupadd docker
• Add a user to docker group: sudo usermod -aG docker $USER
• Activate the changes: newgrp docker

• Check the installation by displaying docker version: docker --version

Exercise 2. Practicing docker commands

This task mainly helps you to learn basic commands used by docker CLI such as run, pull, listing images, attaching data volume, working with exec (like ssh a container), checking IP address, and port forwarding. Docker commands

• Pull an image from Docker Hub and run an Ubuntu container in detached mode and assign your name as container-name, install HTTP server (use docker exec command) and use port forwarding to access container-http traffic via host port 80.
  • Create a login account at Docker Hub sign-up page (Prerequisite as mentioned earlier)
  • Login into your docker account from docker host terminal: docker login
    • Input the following:
      • Username: your docker hub id
      • Password: Docker hub password
  • NB! The login part is not mandatory but needed/recommended since recently docker hub limited the number of docker pull from a particular IP. As you are using the university network through VPN, it serves as a single IP for the Docker hub. (Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit)
  • Pull an image from docker hub: docker pull ubuntu
  • Check the downloaded images in local repository: docker images
  • Run a simple ubuntu container: docker run -dit -p 80:80 --name <new_container_name> ubuntu
    • <new_container_name> = please type any name for that container
  • Get the bash shell of container: docker exec -it <container_name> sh
  • Exit from the container: exit
  • Connect to container and update the apt repo: docker exec -dit <container_name> apt-get update
  • Install http server: docker exec -it <container_name> apt-get install apache2
  • Check the status of http server: docker exec -it <container_name> service apache2 status
  • If not running, start the http server: docker exec -it <container_name> service apache2 start
  • Check the webserver running container host machine: curl localhost:80
  • Check the ip address of the container: docker inspect <container_name> | grep -i "IPAddress"
• Host directory as a data volume: Here you are mounting a host directory in a container and this is useful for testing the applications. For example, you store source code in the host directory and mount it in the container, the code changed in the host directory file can affect the application running in the container.
  • Accessing a host file system on the container with read-only and read/write modes:
    • Create directory with name test:mkdir test && cd test, Create a file:touch abc.txt
    • Run a container with a -v parameter to mount the host directory to the container
      • Read only:docker run -dit -v /home/ubuntu/test/:/home/:ro --name vol1 ubuntu sh
      • Access the file in a container in the path /home and try to create a new file(you should see access denied) from container docker exec -it vol1 sh, cd /home, ls, exit.
      • Read/write:docker run -dit -v /home/ubuntu/test/:/home/:rw --name vol2 ubuntu ,docker exec -it vol2 sh,cd /home, ls,Try to create some text files,exit.You can see the created files in host machine cd /home/ubuntu/test/,ls
    • Stop and delete the container : docker stop vol1,docker rm vol1
• Data volume containers: A popular practice with Docker data sharing is to create a dedicated container that holds all of your persistent shareable data resources, mounting the data inside of it into other containers once created and set up.
  • Create a data volume container and share data between containers.
    • Create a data volume container docker run -dit -v /data --name data-volume ubuntu ,docker exec -it data-volume sh
    • Go to volume and create some files:cd /data && touch file1.txt && touch file2.txt Exit the container exit
    • Run another container and mount the volume as earlier container:docker run -dit --volumes-from data-volume --name data-shared ubuntu,docker exec -it data-shared sh
    • Go to the data directory in the created container and list the files: cd /data && ls
• NB! Complete the below mentioned scenario and take a screenshot of the outputs.
• Scenario : Containerize a program to display your name, hostname, and IP address of the container. Use any of your favorite programming languages. The expected output is to display the information in the terminal.

Exercise 3. Building a Dockerfile

The task is to create your own docker image and try to run the same. More information about Dockerfile (https://docs.docker.com/engine/reference/builder/). A Dockerfile is a text document that contains all the commands a user could call on the command line to assemble an image. Using docker build users can create an automated build that executes several command-line instructions in succession.

Dockerfile commands:

• The scenario is to deploy a web application (flask) to read a IoT sensor data from csv and display on the web page
  • Download the web application using git clone https://github.com/shivupoojar/webaspp-flask.git
  • cd webaspp-flask
  • Create a requirement file to install libraries required to run the Flask application requirements.txt and add the text Flask and pandas in the file.
  • Create Docker file to build a container image using the Dockerfile commands sudo vi Dockerfile

FROM python:3.8-slim-buster
WORKDIR /app 
COPY requirements.txt requirements.txt
RUN pip3 install -r requirements.txt
COPY . .
CMD [ "python3", "-m" , "flask", "run", "--host=0.0.0.0"]

• Build the docker file: docker build -t myflask .
• Check the created image in the repository list:docker images
• Run the container with created image: docker run -P -d -p 5000:5000 myflask
• Check the running container at http://Your_VM_IP:5000/
• NB! Modify the web page to display hostname and IP address along with sensor data. Restart the docker container after you modify the content using docker restart <container_name> command

and take the screenshot of the same.

Exercise 4. Shipping a docker image to Docker hub

Docker hub is a hosted repository service provided by Docker for finding and sharing container images with your team.

• Create a login account in docker hub using this link and sign up.
• Push the docker image to docker hub
  • Initially login into your docker account from docker host terminal: docker login
    • Provide input to the following:
      • Username: your docker hub id
      • Password: Docker hub password
  • Tag your image before pushing into docker hub:docker tag myflask your_docker_username/devops2021:prac2
  • Finally push the image to docker hub: docker push your_docker_username/devops2021:prac2
• Scenario : Export the container as a tar file by using the command docker export.
  • NB! Take the screenshot of the Docker hub account showing your image.

Exercise 5. Visualize sensor data in Grafana: containerizing an application.

In this use case, you are going to perform such tasks as:

• Creating a docker volume.
• Creating a container to download and store the data in the docker volumes.
• Creating a container with Grafana and read the sensor data from the docker volumes and display it using Grafana.

NB! This exercise should be completed by yourself using the knowledge from the previous exercise.

• Creating a docker volume. You may use docker volume create <vol_name> command.
• Create a Dockerfile that should download the sensor data (you can use git clone https://github.com/shivupoojar/devops-prac2.git) and run the container that stores this data in the docker volumes.
  NB! The above task can be done differet ways, following is an example.
  • Use base image as ubuntu
  • Make working directory as /app using WORKDIR
  • Define ARG as arguement variable git_url(You can use git project url https://github.com/shivupoojar/devops-prac2.git as argument while building an image)
  • Update the apt-get
  • Install git using apt
  • Clone the project (use ARG variable containig project url) containing iot sensor data
  • to build image you can use docker build -t image_name --build-arg git_url="https://github.com/shivupoojar/devops-prac2.git" .
  • run the container as docker run --name name -d --mount source=volume_name destination=/app image_name
• Now let us create a Dockerfile for grafana image as shown below and build it, here you should use Grafana docker image and install grafana plugin for reading CSV (for more information please refer to this).

FROM grafana/grafana
RUN grafana-cli plugins install marcusolsson-csv-datasource

• Built the image using doker build command
• Run the container with the following options:
  • use port mapping 3000:3000 (Attach allow-all security group to your VM to access the web app through port 3000)
  • link the docker volume to access sensor data using mount option (can refer here)
• In Grafana, we are using local file path as a data source (.csv file ) for that you modify the grafana.ini configuration file as Local mode
  • To allow the Local mode you need to update grafana.ini file located at /etc/grafana/grafana.ini and to do this, already updated grafana.ini can be found in mounted directory and can be copied to /etc/grafana/grafana.ini using cp command (Use docker exec and command to use is cp path/grafana.ini /etc/grafana/grafana.ini).
  • After doing this, restart the container and again open the grafana dashboard and continue.
• After running the container, you should access the grafana using http://VM_IP:3000 and login using username as admin and Password as admin and skip for password update.
• Now let us add a data source - Click on Configuration tab --> Data Source --> CSV (search for data source as CSV).
  • Enable Local
  • Path --> /usr/share/grafana (this should same as you specify mount name while attaching docker volume to the container)
  • Click on Save and Test
• Creating a dashboard
  • Click on create (+) --> dashboard than Add an empty panel
  • You can refer to this manual to create a dashboard
    • Path-->airdata.csv
    • Fields--> time (time format) and air_Temp_float(Number format)
    • Change the time range 30 days and you should see the data in grafana as shown below (You can design nice dashboard eventhough :-))

• NB! Take the screenshot of the grafana dashboard with clearly visible of VM IP address .

Deliverables

• Upload the screenshot taken wherever mentioned
• Pack the screenshots into a single zip file and upload them through the following submission form.
• Your instance must be terminated!

Deadline: 24 Sept 2021