Cloud Computing 2021/22 spring

Practice 6 - Load balancing

In this lab we will take a look at load balancing web applications running ontop of IaaS cloud instances. You will set up 2 instances: One with a simple webserver application and second one with a load balancer. You will define a load balancing group which will contain your own application server, together with other student servers and generate synthetic user requests against your load balancer using JMeter. The goal is to learn how load balancing works in the cloud, how to set up load generation for simple performance testing and take a look at different load balancing strategies (bonus task).

We will use Python Flask web server with Redis for the application and nginx for the load balancer.

In this lab we are again working on the OpenStack cloud platform, located at: https://stack.cloud.hpc.ut.ee/

• To access the UT cloud resources your computer has to be inside the UT network. So you should set up a VPN connection to university network

In case of issues check:

1. Pinned messages in the #lab6-loadbalancing Slack channel.
2. Possible solutions to common issues section at the end of the guide.
3. Ask in the #lab6-loadbalancing Slack channel.

Exercise 6.1. Setting up an Application Server instance

We will create a new VM instance for hosting the Python-based Web Server. We will download and set up an already implemented Flask application that keeps track of how many users have visited the page and from which IP-s.

• Create a new VM instance from an image (don't use a snapshot!) with the following configuration:
  • Source:
    • Image: ubuntu20.04
    • Make sure to enable the Delete Volume on Instance Delete option
  • Networks: provider_64_net
  • Flavour: m1.xsmall (this time we will run multiple VMs simultaneously, so let's be conservative with resources)
• Make sure that port 80 is open on the VM instance
  • (hint: use Security Groups. Security Groups can be added/removed from instances even when they are running.)
• Log into the instance through SSH
• Install the required software packages on the instance:
  • Refresh the list of software packages: sudo apt update
  • Install the following ubuntu packages: redis-server , python3-pip
• Download the .zip archive containing the web application: lab6app.zip
  • Move the contents of the app to your VM by either:
    • e.g. using scp
    • using wget to download the zip directly to the instance and unzip to unpack it.
      • wget https://courses.cs.ut.ee/2021/cloud/spring/uploads/Main/lab6app.zip

About the Python Application and Redis

This web app checks the IP address of who is making the request and stores/increments counter values in a hashmap, tracking how many times each IP address has visited (key: ip address, value: no. of requests). The HashMap is stored in the Redis in-memory data structure store. In this lab we are simply using Redis as a database, but Redis is actually designed for distributed use, meaning it can be used as a fast cache/database/message broker shared by multiple machines in a cluster.

• Prepare & Start the Flask server
  • First, make sure we have the required python libraries (flask, redis) as described in requirements.txt
    • sudo pip3 install -r requirements.txt
  • Start the server with sudo python3 main.py
    • Note that the Flask app is using port 80. To be able to bind to port 80, we have to run Flask as root user (for other ports, this is not necessary).
• Access your instance through a web browser using its Public IP to check that the website is working as required
• Modify ./templates/index.html to make the web page more personal to you, so that others would recognize that this application server was set up by you when they visit the page.
  • How you decide to modify it up to you, but there should at least be your Full Name present.
  • You may want to make it visually very recognizable to differentiate between your server and other students servers more easily.
  • nano is a suitable command line file editor to modify text files.
• If at any point you need to clear your entire Redis storage, you can run this command:
  • redis-cli FLUSHDB
• Take a screenshot of your application server web page

Exercise 6.2. Setting up load balancer instance and balancing multiple application servers

In this exercise, we set up another instance and install a NginX load balancer on it.

A Load balancer (LB) distributes all user requests to a web page across multiple application servers. Load balancers are used to increase capacity (concurrent users) and reliability of applications and are a required component for auto-scaling: meaning changing the number of application servers dynamically based on number of active users.

(image taken from http://www.netdigix.com/linux-loadbalancing.php)

• We are using Nginx (http://nginx.org/en/) as a load balancer.
• Create a new instance (Should also be Ubuntu 20.04) for the load balancer
  • This instance should also have the port 80 open, just like with the first instance
  • Make sure to select the "Delete Volume on Instance Delete" option in the Source tab
  • Use m1.xsmall flavor
• Install Nginx on the instance
  • Refresh the list of software packages: sudo apt update
  • Install the following ubuntu package: nginx-full
• Modify Nginx configuration to add your application server IP into the list of managed services
  • Download the example nginx load balancer configuration: wget https://courses.cs.ut.ee/LTAT.06.008/2021_spring/uploads/Main/default
  • Modify the downloaded default configuration file
    • Find the upstream ________ block
    • Modify the server 172.17.64.109; line in the configuration file and replace the existing IP (172.17.64.109) with the actual IP of your application server.
• Overwrite the default nginx site configuration on the instance with the modified configuration
  • sudo cp default /etc/nginx/sites-enabled/default
  • Remember to run this command again every time you modify the local default file
• Reload Nginx service
  • sudo service nginx reload
  • Remember to run this command again every time you modify the local default file
• Visit the IP address of the load balancer using a web browser and verify that it displays your application server
• Now also add lab instructors and few other student's application servers under your load balancer to distribute user requests between multiple servers.
  • Create a new server IP_ADDRESS; line inside the upstream ________ block for each additional server you wish to add
  • Lab instructor Application Server IP is: 172.17.66.4
• Take a screenshot of visiting your application server web page through the load balancer (Browser is accessing load balancer IP address, but your application server content is shown)

Exercise 6.3. Verifying that the load balancing is working properly

Lets test whether the previous steps have been completed successfully by verifying that the load balancing is working and user traffic is being rerouted into your application server.

• Visit the load balancer multiple times. Do you recognize your own application sever?
• Ask other students to visit your load balancer or to add your application server into their load balancer
• Wait until you start seeing requests on your application server tracker from a number of other locations.
• You can check current incoming HTTP connections to your Application Server using the following command: netstat | grep http
  • Take a screenshot of the output of netstat command. Try to have it display more than 3 connections when other student`s requests are being redirected to your server.
  • If you have issues getting sufficient number of connections to show up, You may try generating sufficient connections yourself. It may also be best to finish the next exercise first before taking this screenshot, as it will allow you to generate a large number of synthetic user requests.

Exercise 6.4. Generating additional user traffic for benchmarking

Lets check whether the load balancer can now handle a higher number of simultaneous user requests.

• Your task is to generate a large number of user requests to the load balancer and verify how many of those requests are sent to your application server by the load balancer.
• To generate a large number of user request, we will use Apache JMeter, which is a Java based framework for simulating users and generating web traffic.
• Investigate how to set up JMeter load generation as web requests
  • A nice tutorial for creating web testing configuration in JMeter is available here: https://jmeter.apache.org/usermanual/build-web-test-plan.html
• Download JMeter.
  • https://jmeter.apache.org/
• Run JMeter
  • It is a Java application, either:
    • Use one of the scripts bundled with it inside the "bin" folder, such as jmeter.bat for Windows or the executable script file jmeter for Linux.
    • Start it by opening bin/ApacheJMeter.jar file with Java directly, .e.g. java -jar ApacheJMeter.jar
• Set up web testing configuration
  1. Right click on Test plan -> Add -> Config Element->HTTP Request Defaults configuration
     • Server Name or IP: use IP of your LOAD BALANCER instance
  2. Right click on Test plan and add Thread Group
     • Some of the suggested parameters:
       • Number of threads/users: 10 (you can use larger number of you wish)
       • Loop count: how many requests you want each user to perform.
  3. Right click on Thread Group and add HTTP request sampler
     • Set path to /
  4. Right click on Thread Group and add Response Time graph listener
     • Configure Graph interval to 1000 ms
• Make sure your Load balancer has more than one application server listed in the Nginx config (as we did in task 6.2). They may be the instrcutors server or other students servers.
• Generate 2000 requests against your load balancer using JMeter.
• Note down how many of those requests end up on your application server. How large percent of your generated user requests ended up visiting your server?
  • Take a screenshot of the JMeter tool configuration (HTTP request element) which you used for load generation.
  • Take a screenshot of the JMeter result visualization view (Response Time graph element) which displays the results.
  • Tips
    • The "Display Graph" button will show you the graph.
    • To clear any previous results ( including for the graph) select in top-menu: Run > Clear

Bonus task

Your task is to create another Application server, this time a bit more capable than the first one, add both application servers to your load balancer and investigate the performance of these two instance types and also the effect of different load balancing strategies.

• Create another Applications server instance of m2.tiny flavor, just like in Exercise 6.1.
• Add the Application server into your load balancer configuration. And remove other userappliaction servers if you added any in the previous exercises.
• Your load balancer should now manage only two application servers:
  1. m1.xsmall instance, your first application server you made in 6.1
  2. m2.tiny instance, your second application server
• Our initial Python script is very fast, which makes it difficult to gauge the respective capabilities of these two instance types. Lets add a bit more computation heavy task to our scripts
  • Add some Pi calculation code at the end of your Python application on BOTH application servers
  • You can find a Python implementation for computing PI from here:
    • http://www.codecodex.com/wiki/Calculate_digits_of_pi#Python (Scroll to very end of Python section for a more concise version)
    • Add the Pi calculation code to the main request handling part of the .py file (the home() function ). Also update the index.html template file so that it outputs the calculated value of Pi as well (you need to pass the value to the template as an argument to the render_template() at the end of home() ) .
    • As a result, your application should now also print out a very long number which represents the first X digits of Pi.
• Configure Nginx to use the Least connections load balancing strategy, generate 2500 user requests using JMeter to your load balancer and track how many of these 2500 user requests end up in application server 1 vs application server 2
  • You will find description of available load balancing strategies here: http://nginx.org/en/docs/http/load_balancing.html
• Deliverables :
  1. Answer: When using the least_conn; load balancing strategy, what percent of user request are sent to first and second application server?
     • Also provide screenshots from both JMeter and your application tracker to display the state after your experiments.
  2. Answer: Which of these instances are more capable in terms of responding to user requests?
     • Take a screenshot of the top command line command output on both application server instances while the requests are being sent to the instances.
     • Answer: What is the CPU load value on both instances? What does this indicate in regards to the performance of these cloud instance types?

Deliverables

• DO NOT leave your instances running after submitting the solution. This lab requires quite a few VM CPU cores per student and may mean some students can not start lab exercises until previous students shut down their instances.
• Submit screenshots from the exercise:
  1. Exercise 6.1: Take a screenshot of your application server web page
  2. Exercise 6.2: Take a screenshot of visiting your application server web page through the load balancer
  3. Exercise 6.3: Take a screenshot of the output of netstat command.
  4. Exercise 6.4:
     • Take a screenshots of the JMeter tool configuration (HTTP Request defaults and HTTP Request under thread group) which you used for load generation.
     • Take a screenshot of the JMeter result visualization view (Response Time graph element) which displays the results.

In case of issues, check these potential solutions to common issues:

1. If you can not access your cloud instances over SSH:
   • To access the local university cloud resources your computer has to be inside the Institute network. So you should set up a VPN connection to university network.
   • VPN (English on the right side column) - https://wiki.ut.ee/pages/viewpage.action?pageId=17105590
2. You might encounter an error stating that something is locked that is Ubuntu running some updates in background so please give it few minutes to complete and try again later, if still no luck ask help from lab instructor.
3. If you get errors about permissions when running command line commands inside the instance:
   • In several exercises you are asked to modify, delete or edit files outside your home folder.
   • You will have to use sudo command to elevate file manipulation command permissions to be able to complete such operations.
   • For example: sudo nano /etc/nginx/sites-enabled/default will run nano command under root user with elevated permissions.
   • NB! But be careful, not everything should be run through sudo!