Pilvetehnoloogia 2020/21 kevad

Practice 3 - Working with Heroku Platform as a Service

In this practice session, we will look into Heroku platform as a service. We will explore the Heroku CLI to deploy an application on Heroku platform.

Heroku supports applications written in Ruby, Node.js, Java, Python, Clojure, Scala, Go and PHP, where users can create, deploy and scale the applications. It also supports testing and deploying apps locally and then pushing them into the Heroku environment.

The overall architecture of Heroku:

Figure source link: https://www.oreilly.com/library/view/heroku-up-and/9781449341381/ch02.html

• Dynos: All Heroku applications run in a collection of lightweight Linux containers called dynos. (For more information refer: https://devcenter.heroku.com/articles/dynos)
• Routers: The Heroku router is a piece of software (written using Erlang, called Hermes), which acts as a gateway between your users and the dynos that are running your code.
• Logplex: The Logplex is an open-source tool that replaces the logfile concept by providing a constant stream of information flowing from your application with no beginning and no end.
• Add-ons: Set of other services provided or developed by other service providers such databases and so on belong to Add-on services. One of the most popular add-ons is one provided by Heroku itself. Called Heroku Postgres, this add-on provides PostgreSQL services to all applications.

For more information, please refer to the following links: https://devcenter.heroku.com/categories/reference#

Applications can be deployed on to Heroku in three ways:

• Using Heroku CLI (We are using this method in Exercise 3.2)
• Github based
• Using docker images

The glossary of keywords need to be familiar while using Heroku:

• App: An application that runs on heroku using a set of dynos and has unique URL .herokuapp.com
• Dyno: A container that runs a Heroku app’s code. When a dyno starts up, it runs a single command that is usually specified in the app’s Procfile.
• Free dyno: A dyno that enables to host your app freely on the Heroku platform.
• Web dyno: A dyno that receives HTTP traffic.
• Worker dyno: A dyno does not receive HTTP traffic.
• Procfile: A plaintext file that declares the commands that an app’s dynos run when they startup.
• Heroku CLI: The command-line interface for interacting with Heroku apps.
• Heroku Dashboard: The web interface for interacting with Heroku apps. Available at dashboard.heroku.com.

References

• Heroku documentation: https://devcenter.heroku.com/categories/reference
• Python Flask documentation: https://flask.palletsprojects.com/en/1.1.x/
• Using Python Flask and SQLAlchemy together: https://flask-sqlalchemy.palletsprojects.com/en/2.x/

Exercise 3.1. Getting familiar with the Heroku platform and setting up Heroku CLI

In this exercise, you will learn how to create an account in the Heroku platform and get familiar with the Heroku dashboard. Apart from this, you are going to set up Heroku CLI for application deployment.

• Create a free Heroku account using the link https://signup.heroku.com/dc
• Now, Sign in to the Heroku dashboard and create an application to get familiar with basic components.
  • Click on New--> Create application with name of your choice as shown in figure below

• Open your app and get familiar with the application dashboard.
• Now, click on the Deploy tab, which shows the different modes of deploying the app to the Heroku platform.

• Setting up Heroku CLI
  • Install Python 3.7 or higher on your computer.
    • Linux: https://docs.python-guide.org/starting/install3/linux/
    • Windows: https://docs.python.org/3/using/windows.html
    • Mac - OSx: https://www.python.org/downloads/mac-osx/
  • Installation of Heroku CLI: https://devcenter.heroku.com/articles/getting-started-with-python#set-up
  • After completing the above steps, login into Heroku CLI
    • In Windows: Open Powershell and enter heroku login and follow subsequent steps.
    • In Linux/OSx: Open terminal and enter heroku login.

You should see the login successful as shown below

Exercise 3.2. Creating and deploying python flask application using Heroku CLI

This task mainly helps you to learn the deployment of python applications locally and on the Heroku platform using Heroku CLI.

• Create a python flask application to display your name on the web page and count the visitors.
  • Make a directory with the name heroku-flask and create a subdirectory templates.
  • Create index.html in templates directory and write an HTML code to display your name and number of visitor count. Sample can be downloaded here index.html
  • Create a python program app.py which acts as a backend for your flask application.
    • import flask library

  from flask import Flask, render_template
  import os
  

  • Create flask application

  app = Flask(__name__)
  

  • Define route path for HTTP invocation

  @app.route("/")
  

  • Define method def index() to count the number of visits and to display your name. Here we will use a text file count.txt to store the visitor count and assign that value to variable count and the same value can be displayed on the web page. Create a text file count.txt in the directory and write 0 in the first line of the file. The sample is here count.txt
    • add a logic to read a current count value from count.txt and increment the count value

  def index():
          line = open("count.txt", "r")
          count = int(line.read())
          line.close()
          count+=1
  

  • update the value in count.txt

      line = open("count.txt", "w")
      line.write(str(count))
      line.close()
  

  • return the rendered HTML with count value

      render_template("index.html", count=count)
  

  • Write a main method

      if __name__ == "__main__":
      app.run()
  

The final code looks like this: app.py

• Create Procfile that contains the command to run when the app is deployed. Add the following code web: gunicorn app:app to the Procfile. Here, gunicorn indicates the web interface to use. The app indicates app.py(your flask application service). Sample Procfile can be downloaded hereProcfile
• Create a requirements.txt file in the directory and add the required python packages to be used in your application. Add the following set of packages to the file or download the requirements.txt from here requirements.txt.

click==7.1.2
Flask==1.1.2
gunicorn==20.0.4
itsdangerous==1.1.0
Jinja2==2.11.2
MarkupSafe==1.1.1
Werkzeug==1.0.1 

After completing all the steps. the directory structure looks like,

• Local deployment
  • Now run the following command to deploy the Heroku app locally
    • Install the libraries defined in requirements.txt using pip: pip3 install -r requirements.txt
    • In Windows
      • create Procfile.windows
      • Its content should be: web: python app.py
      • Run: heroku local web -f Procfile.windows
    • In Linux : heroku local

You can open the local app in the browser using the link http://127.0.0.1:5000

• Make a screenshot of the browser showing your deployed application locally
  • NB! Your name must be clearly visible on the screenshot!

• Deploying on Heroku platform

Here, you will learn how to deploy the flask application into the Heroku platform using Heroku CLI. Heroku manages the application code in Heroku's GitHub repository. Each application will have its own repository and created when we create the application in Heroku CLI.

• Create an application using CLI with your name as app nameheroku create yourname --buildpack heroku/python (Make sure that you have logged in using heroku login and you are in heroku-flask application directory)

• Initialize the git git init
• Add current project code to github git add .
• Commit the code git commit -m 'Deploy'
• Attach your code to remote app heroku git:remote -a appname
• Deploy application git push heroku master
• Visualize the app heroku open

• You can check the logs heroku logs --tail
• Make a screenshot of the browser showing your deployed application
  • NB! Your app-name must be clearly visible on the screenshot!

Exercise 3.3. Working with datastore on Heroku platform

In this task, you will learn about working with datastore by provisioning managed Postgres SQL provided by Heroku and how to use this datastore in flask application.

• Provision Postgress SQL service using the command heroku addons:create heroku-postgresql:hobby-dev --app appname, appname--> your application name, the hobby-dev indicates postgress license and we are using a free license in this experiment. For more information refer the link https://devcenter.heroku.com/articles/heroku-postgresql#understanding-heroku-postgres-plans.
• Update the flask application from 3.2 to work with Postgres datastore. For this purpose, we will design a message storeroom application by just modifying few files. We will store and retrieve messages stored by the end-user.
  • Modify index.html to with a text box and button to store the message in the datastore and also need to display the stored messages.

  <html>
    <h1>Hello!! Yourname</h1>
        <form action="/submit" method="POST">
           <div class="form-group">
            <h3>Message</h3>
            <textarea  name="comments"  id=""  cols="30"  rows="10" placeholder="Type a message">
            </textarea>
          </div>
          <input type="submit" value="Submit" class="btn" />
        </form>
    <h1>List of stored Messages</h1>
     <TABLE>
        {% for feedback in results %}
          <tr>
              <td>{{feedback.comments}}</td>
          </tr>
      {% endfor %}
        </TABLE>
     </body>
  </html>
  
• Explanation about the HTML Jinja Template
  • Jinja allows us to interact with data variables that are passed to it (as we did in app.py render_template for the feedback variable) and conveniently interweave them with the HTML.
  • In Jinja, you can access properties of the variables, iterate over them, re-format them, hide parts of the HTML based on boolean expressions etc. See here for more information
• The updated page should now look like this:

• Modify app.py to store and retrieve the messages from Postgress data store. We will use SQLAlchemy to connect with Postgress and more information about SQLAlchemy can found here https://www.sqlalchemy.org/.

The app.py should be modified with the following logic:

• Define datastore connection string for SQLALCHEMY.

  from flask import Flask, render_template, request, redirect
  from flask_sqlalchemy import SQLAlchemy
  import os
  app = Flask(__name__)
  app.config['SQLALCHEMY_DATABASE_URI'] = os.environ['DATABASE_URL']
  db = SQLAlchemy(app)
  

  • When testing locally, you can check the connection string by running the command heroku config --app appname and writing it into the DATABASE_URL environment variable before you run the applciation locally
• Define a class to initialize and create a Table and its attributes

  class Feedback(db.Model):
      __tablename__ = 'feedback'
      id = db.Column(db.Integer, primary_key=True)
      comments = db.Column(db.Text())
      def __init__(self, comments):
         self.comments = comments
  
  db.create_all()
  db.session.commit() 
  
• Define the initial route path of application

  @app.route("/")
  def index():
      feedbacks = Feedback.query.all()
      return render_template('index.html', results=feedbacks)
  
• Define a method to receive the request when a user submits a message

  @app.route('/submit',methods = ['POST'])
  def submit():
      if request.method == 'POST':
          comments = request.form['comments']
          data = Feedback(comments)
          db.session.add(data)
          db.session.commit()
      return redirect("/", code=302)
  
• PS! Do not forget you also need the Python script to start the app in the main part of your Python script (check the example in the first exercise)
• Update the requirements.txt file with the following:

autopep8==1.4.4
Click==7.0
Flask==1.1.1
Flask-SQLAlchemy==2.4.0
gunicorn==19.9.0
itsdangerous==1.1.0
Jinja2==2.10.1
MarkupSafe==1.1.1
psycopg2-binary
pycodestyle==2.5.0
SQLAlchemy==1.3.7
Werkzeug==1.0.1  

• Now deploy the application.

git add .
git commit -m 'Deploy'
git push heroku master

• Now you can use heroku open command to automatically open the browser window with a correct address of your application
  • the output should look something like:

• If you want to test locally (it is ok to test only in heroku cloud), you need to define the value of the DATABASE_URL environment variable, which defines where the database is located and its authentication details.
  • You can use heroku config --app appname command to print out the database connection string
  • Define environment variable before deploying your application locally:
    • in Windows CMD command line: set DATABASE_URL=postgres://yrt...
    • In windows PowerShell: $env:DATABASE_URL="postgres://yr..."
    • in Linux: export DATABASE_URL=postgres://yr...
• Make a screenshot of the browser showing your deployed application
  • NB! Your app-name must be clearly visible on the screenshot!

Exercise 3.4. Improving the Heroku application

For this task, you have to design & implement some improvements to the application we just created. You should modify the application to ask for the name of the user and to track when new messages are posted. It also make sense to limit how many messages are shown on the main page.

1. Modify the Feedback database class to also include time of the message.
   • Check SQLAlchemy documentation and other online resources how to add datetime field to the database class.
     • It may also make sense to have the current date automatically generated in database.
   • The application should display when the messages were created on the main page
   • Modifying SQL table structure is somewhat complex in SQLAlchemy. But it is simple to delete the table (NB! and all data!!) and recreate it with new structure (After modifying the Feedback class) by running the drop table feedback SQL command.

   db.engine.execute("drop table feedback")
   db.session.commit()
   

   • NB! remember to remove this line once you are done with the database change!
   • You can also run it from the Python console instead to be safer:

   import app
   app.db.engine.execute("drop table feedback")
   app.db.session.commit()
   

2. Modify the application to limit the display to only 10 latest messages.
   • Check SQLAlchemy documentation how to limit the number of entries returned by database queries (Feedback.query.all()) and how to order them.
3. Modify the application to also ask name of the user who is posting messages
   • You need to modify the HTML form, submit() method and the Feedback class.
   • The application should also display the user name next to each of the messages on the main page
   • The final output looks like

• Make a screenshot of the browser showing your deployed application
  • NB! Your app-name must be clearly visible on the screenshot!

Bonus tasks

Bonus task 1: User authentication

The goal is to require user to be logged into the application to post messages.

1. Update the application with authentication mechanism
   • Should be simple username and password-based or oAuth based.
2. Having an account registration page is not required
   • However, in that case, make sure to also provide a set of login details for the TA's to log into your application

Bonus task Deliverables:

1. Code & link to your Heroku app
2. Screenshots of displaying how the authentication works.

Deliverables

• Any screenshots you were asked to take during the exercises
• Application code source files (.py, .html, requirements.txt)
  • Make sure Do NOT provide the postgre connection string!! It includes the username and password to connect to you your Heroku postgre service.
• Write your Heroku application URL (As a comment or as readme,txt file).
• Pack the screenshots and the code into a single zip file and upload them through the following submission form.