Cloud Computing 2021/22 spring

Practice 3 - Working with Heroku Platform as a Service

In this practice session, we will look into Heroku platform as a service. We will explore the Heroku Command-Line Interface (CLI) to deploy an application on Heroku platform.

Heroku supports applications written in Ruby, Node.js, Java, Python, Clojure, Scala, Go and PHP, where users can create, deploy and scale the applications. It also supports testing and deploying apps locally and then pushing them into the Heroku environment.

The overall architecture of Heroku:

Figure source link: https://www.oreilly.com/library/view/heroku-up-and/9781449341381/ch02.html

• Dynos: All Heroku applications run in a collection of lightweight Linux containers called dynos. (For more information refer: https://devcenter.heroku.com/articles/dynos)
• Routers: The Heroku router is a piece of software (written using Erlang, called Hermes) which acts as a gateway between your users and the dynos that are running your code.
• Logplex: The Logplex is an open-source tool that replaces the logfile concept by providing a constant stream of information flowing from your application with no beginning and no end.
• Add-ons: Set of other services provided or developed by other service providers such databases and so on belong to Add-on services. One of the most popular add-ons is one provided by Heroku itself. Called Heroku Postgres, this add-on provides PostgreSQL services to all applications.

For more information, please refer to the following links: https://devcenter.heroku.com/categories/reference#

Applications can be deployed on to Heroku in three ways:

• Using Heroku CLI (We are using this method in Exercise 3.2)
• Github based
• Using docker images

The glossary of keywords need to be familiar while using Heroku:

• App: An application that runs on heroku using a set of dynos and has unique URL .herokuapp.com
• Dyno: A container that runs a Heroku app’s code. When a dyno starts up, it runs a single command that is usually specified in the app’s Procfile.
• Free dyno: A dyno that enables to host your app freely on the Heroku platform.
• Web dyno: A dyno that receives HTTP traffic.
• Worker dyno: A dyno does not receive HTTP traffic.
• Procfile: A plaintext file that declares the commands that an app's dynos run when they startup.
• Heroku CLI: The command-line interface for interacting with Heroku apps.
• Heroku Dashboard: The web interface for interacting with Heroku apps. Available at dashboard.heroku.com.

References

• Heroku documentation: https://devcenter.heroku.com/categories/reference
• Using Python Flask and SQLAlchemy together: https://flask-sqlalchemy.palletsprojects.com/en/2.x/

Exercise 3.1. Getting familiar with the Heroku platform and setting up Heroku CLI

In this exercise, you will learn how to create an account in the Heroku platform and get familiar with the Heroku dashboard. Apart from this, you are going to set up Heroku CLI for application deployment.

• Create a free Heroku account using the link https://signup.heroku.com/dc
• Now, Sign in to the Heroku dashboard and create an application to get familiar with basic components.
  • Click on New -> Create new app, and create an application with the name : yourname-v1 as shown in figure below

• Open your app and get familiar with the application dashboard.
• Now, click on the Deploy tab, which shows the different modes of deploying the app to the Heroku platform. We will work with Heroku Git-based deployment from the CLI.

• Setting up Heroku CLI
  • Install Python 3.7 or higher on your computer.
    • Linux: https://docs.python-guide.org/starting/install3/linux/
    • Windows: https://www.python.org/downloads/windows/
    • Mac - OSx: https://www.python.org/downloads/mac-osx/
  • Installation Heroku CLI: https://devcenter.heroku.com/articles/heroku-cli#install-the-heroku-cli
    • Note that installing Git is also needed for Heroku CLI!
  • After completing the above steps, login into Heroku CLI
    • In Windows: Open Powershell and enter heroku login and follow subsequent steps.
    • In Linux/OSx: Open terminal and enter heroku login.

You should see the login successful as shown below

Exercise 3.2. Creating and deploying python flask application using Heroku CLI

This task mainly helps you to learn the deployment of python applications locally and on the Heroku platform using Heroku CLI. We will deploy our text-file based Flask message board application.

• Fetch the application source code
  • git clone https://bitbucket.org/jaks6/cloud-computing-2022-lab-1.git lab1app
• To make the application work in Heroku, we need to create a Procfile. The Procfile will instruct Heroku how exactly this application should be run (e.g. what command starts this software, are additional arguments necessary, etc)
• Go inside the application's src directory (containing app.py and other files) and create a file called Procfile
  • Contents of Procfile should be web: gunicorn app:app.
    • web indicates the process type. In case of "web" type, Heroku routers will forward HTTP traffic to this process.
    • gunicorn indicates the web interface to use. Gunicorn is a Web Server Gateway Interface. It acts as a lightweight server who delegates incoming requests to the actual Python server ( based on Flask in our case). We use Gunicorn, because it allows to handle multipel requests concurrently, by default a Flask will process only handle 1 request at a time. Read here for more on why we use Gunicorn with Heroku.
    • app:app indicates which application gunicorn should use - "app:app" refers to module name and filename ( module app, file app.py, your flask application).
• Update the requirements.txt, adding Gunicorn to the list of packages.
  • gunicorn==20.1.0

Local deployment

Now let's run the application locally (on our own development machine instead of Cloud) using Heroku and Gunicorn. Gunicorn actually isn't available on Windows, so in case of using Windows, we will use an alternative Procfile.

• First, install the libraries defined in requirements.txt using pip in a venv:
  • On Windows:
    • py -m venv env or python3 -m venv env
    • .\env\Scripts\activate
    • pip install -r requirements.txt
  • On Linux:
    • python3 -m venv env
    • source env/bin/activate
    • pip3 install -r requirements.txt
  • Then, run the app with Heroku (adjusting for Windows OS, if necessary):
    • In Windows
      • create a file Procfile.windows with contents: web: flask run
      • Run: heroku local -f Procfile.windows
      • We are using an alternate Procfile which runs the app with python directly (like we did in lab 1)
    • In Linux : heroku local

You can open the local app in the browser using the link http://127.0.0.1:5000

Deploying on Heroku platform

Here, you will learn how to deploy the flask application into the Heroku platform using Heroku CLI and Git. Heroku manages the application code in Heroku's Git repository. Each created Heroku application will have its own git repository.

• Update the current git project by adding Heroku's repository as a remote heroku git:remote -a appname.
  • appname - the name of the application you created on Heroku in the 1st task
  • This allows us to start deploying updates to the code to Heroku by pushing git commits.
• Add current changes code changes to git git add .
• Commit the code git commit -m "Added Heroku Procfile and Gunicorn"
• Deploy application git push heroku master
  • You should start seeing the application being deployed - requirements being installed, etc.

• Open the deployed app heroku open, make sure everything works!
  • You can check the logs heroku logs --tail
• Provide the exact URL (...herokuapp.com) of your app as a deliverable
• Make a screenshot of the browser showing your deployed application
  • NB! Your app-name (URL) must be clearly visible on the screenshot!
• Experiment 1: Perform the following experiment:
  • Enter a few messages to the app, then let it run idle for at least 30 minutes (nobody should load the page for at least 30 minutes). After at least 30 min has passed, access the app again. Answer the below question:
    • What happened to the state / data of the app? Why did this happen?
    • Note: if you think somebody else has somehow accessed your app within the time, you can use heroku logs to double-check if the app really was idle for 30 minutes or not.

Exercise 3.3. Working with datastore on Heroku platform

In this task, you will learn about working with datastore by provisioning managed Postgres SQL provided by Heroku and how to use this datastore with the database version of our application.

• Exit the previous tasks Flask application directory, and fetch the database version of the code (the one introduced in lab 2). This version of the app uses SQLAlchemy to connect with PostgreSQL, more information about SQLAlchemy can found here https://www.sqlalchemy.org/. :
  • git clone https://bitbucket.org/shivupoojar87/task3lab2app.git lab2app
  • Enter the directory of lab2app
  • Create a 2nd Heroku application, this time using CLI instead of the web dashboard. As app name use yourname-v2 heroku create yourname-v2 --buildpack heroku/python (Make sure that you have logged in using heroku login and you are in lab2app application directory)

• Add a Procfile, and add gunicorn to requirements.txt (just like last exercise)

• Provision Postgres SQL service using the command heroku addons:create heroku-postgresql:hobby-dev --app appname, appname--> your application name
  • This creates a postgres database service running in Heroku for our app.
  • hobby-dev indicates postgres license and we are using a free license in this experiment. For more information refer here.
  • deliverable: take a screenshot of the output of heroku addons command
• If you inspect the code of app.py, you will see that the application is configured to use a database specified by the environment variable "DATABASE_URL":

  app.config['SQLALCHEMY_DATABASE_URI'] = os.environ['DATABASE_URL']

  • DATABASE_URL defines where the database is located and its authentication details.
  • In lab 2 with Docker, we passed the value of DATABASE_URL when starting the container. With Heroku, because we have added the Postgres addon, this value is automatically defined on the Heroku platform.
  • You can inspect the value of DATABASE_URL using heroku config. You will need this value to try running the database-version locally.
• Let's try running this version of the application.
  • heroku git:remote -a appname
  • git add .
  • git commit -m 'Deploy'
  • git push heroku master
• Now you can use heroku open command to automatically open the browser window with a correct address of your application
• If you experience error such as UniqueViolation: duplicate key value violates unique constraint "pg_type_typname_nsp_index, try running heroku restart or simply wait a few minutes. Sometimes applying database changes can take time.
• You can also explore your Cloud database at https://data.heroku.com/
• If you want to test locally (it is OK to test only in Heroku cloud), you need to define the value of the DATABASE_URL environment variable, which defines where the database is located and its authentication details.
  • Define environment variable before deploying your application locally:
    • in Windows CMD command line: set DATABASE_URL=postgres://yrt...
    • In windows PowerShell: $env:DATABASE_URL="postgres://yr..."
    • in Linux: export DATABASE_URL=postgres://yr...
• Make screenshots of the heroku addons and heroku config commands after completing the above steps successfully.
  • NB! The DATABASE_URL value includes not only the DB address but also your username and password (the part between postgres:// and @ ). MAKE SURE TO NO SHARE THIS WITH US AS A GOOD PRACTICE OF NOT FREELY DISTRIBUTING CREDENTIALS. You can just draw over that part of the screenshot.
  • NB! The entire output of those commands and your app-name must be clearly visible in the screenshots!
• Experiment 2: Repeat the same test as earlier: post a few messages on the Heroku deployed version (not local), then leave the app idle for at least 30 min, then re-visit the application. Answer these questions
  • How is the behavior different compared to the text-based version? Why do you think this is happening?

Exercise 3.4. Improving the Heroku application

For this task, you have to design & implement some improvements to the application we just created. You should modify the application to ask for the name of the user and to limit how many messages are shown on the main page.

1. Modify the Messageboard database class to add a "Author" (name) field to the database table.
   • Study the existing code and check SQLAlchemy documentation and other online resources how to add extra fields to the database table.
   • You need to modify the HTML form, home() method and the Messageboard class.
   • The application should display the author of the message next to the contents on the main page
   • Modifying SQL table structure is somewhat complex in SQLAlchemy. But it is simple to delete the table (NB! and all data!!) and recreate it with new structure (After modifying the Messageboard class) by running the drop table messages SQL command.

   db.engine.execute("drop table messages")
   db.session.commit()
   

   • NB! remember to remove this line once you are done with the database change!
   • You can also run it from the Python console instead to be safer:

   import app
   app.db.engine.execute("drop table messages")
   app.db.session.commit()
   
2. Modify the application to limit the display to only 10 latest messages.
   • Check SQLAlchemy documentation how to limit the number of entries returned by database queries (Messageboard.query.all()) and how to order them.
   • The limiting should be done on the Database/SQLAlchemy level, when querying, NOT with with something like Python list manipulation operations or Jinja template operations (this was done in Lab 1 home.html).
   • The final output looks something like this (your design can vary):

• Make a screenshot of the browser showing your deployed application
  • NB! Your app-name must be clearly visible on the screenshot!

Bonus tasks

Bonus task 1: User authentication

The goal is to require user to be logged into the application to post messages.

1. Update the application with authentication mechanism
   • Should be simple username and password-based or oAuth based.
2. Having an account registration page is not required
   • However, in that case, make sure to also provide a set of login details for the TA's to log into your application

Bonus task Deliverables:

1. Code & link to your Heroku app
2. Screenshots of displaying how the authentication works.

Deliverables

• Any screenshots you were asked to take during the exercises
• Text answers to questions regarding Experiments 1-2
• Application code source files for the v1 and v2 of the app (.py, .html, requirements.txt, Procfiles)
  • Make sure Do NOT provide the username and password within your Postgres connection address.
• Provide both your Heroku application URLs (As a comment or as readme.txt file).
• Pack the screenshots and the code into a single zip file and upload them through the following submission form.