Practice 14 - Cloud service modeling

In this Practice session, you will learn about cloud service deployment modeling using TOSCA, Winery, and xOpera. You will learn how to create service templates using TOSCA in Winery graphical editor. Also, you will learn how to automatically deploy the created models using an orchestration engine known as xOpera.

The lab content is related to the RADON Horizon 2020 EU project (http://radon-h2020.eu/), which goal is to unlock the benefits of Serverless FaaS for the European software industry using TOSCA language and developed tools.

• G. Casale, M. Artac, W.-J. van den Heuvel, A. van Hoorn, P. Jakovits, F. Leymann, M. Long, V. Papanikolaou, D. Presenza, A. Russo, S. N. Srirama, D.A. Tamburri, M. Wurster, L. Zhu RADON: rational decomposition and orchestration for serverless computing. SICS Software-Intensive Cyber-Physical Systems. 2019 Jan 1-11. https://doi.org/10.1007/s00450-019-00413-w

References

1. Azure Cloud documentation: https://docs.microsoft.com/en-us/azure/?product=featured
2. TOSCA: http://docs.oasis-open.org/tosca/TOSCA-Simple-Profile-YAML/v1.3/TOSCA-Simple-Profile-YAML-v1.3.html
3. xOpera documentation: https://xlab-si.github.io/xopera-docs/
4. Winery documentation https://winery.readthedocs.io/en/latest/user/yml/index.html

NB! Before you start, you are tasked with an additional responsibility to measure how long it takes you to solve exercises 14.2 and 14.4 (separately).

• Note down when you start and when you finish solving those exercises.
• No need to try to rush, we would like to know the average time it takes to design Serverless functions using Winery and deploy them using xOpera.
• We will note down when you need to start or stop measuring time.

Exercise 14.1 Setting up RADON Graphical Modelling tool: Winery

In this task we will set up a new OpenStack VM and install RADON Winery graphical modeling tool inside it.

1. Launch an OpenStack VM for this lab.
   • Use your last name as a part of the instance name!
   • Source: Use Instance Snapshot, choose CC Ubuntu 18.04 with Docker 20.10.6
     • in this Ubuntu-based snapshot, the installation of Docker (as we did in Lab 2) has already been done for us. We will use it to deploy a Winery Docker container.
     • Enable "Delete Volume on Instance Delete"
   • Flavour: should be m2.tiny
2. Create a Docker container with the Winery image from Docker Hub using the following CLI command:

   • docker run -itd -p 8080:8080 \
       -e PUBLIC_HOSTNAME=localhost \
       -e WINERY_FEATURE_RADON=true \
       -e WINERY_REPOSITORY_PROVIDER=yaml \
       -e WINERY_REPOSITORY_URL=https://github.com/UT-Cloud-Computing-Course/radon-particles \
       opentosca/radon-gmt

3. You can access the Winery web interface using your browser on the URL:
   http://IP_OF_OPENSTACK_VM:8080
   • Replace the IP_OF_OPENSTACK_VM with the actual OpenStack VM IP ).

Exercise 14.2 Designing a service template in Winery

In this task, we will design a new Cloud deployment service template. We will try to design a service model for deploying the exact same Azure FaaS function we used last week and to automate the manual steps we did inside the Azure platform.

We will be using TOSCA models created in the RADON project, which are individually deployable and freely composable software service blocks that contain Ansible playbooks for installing, starting, configuring, stopping, and deleting them. There are models for AWS, Azure, Google Cloud, etc.

• The source code for these models is available here: http://github.com/radon-h2020/radon-particles
• We are using a modified fork for this course: https://github.com/UT-Cloud-Computing-Course/radon-particles

NB! Note down the time when you have started solving this task (14.2)

NB! Best to avoid spaces in names or properties while you configure things in this Task.

Create a new Service template:

Define the name of the service template. Also, make sure to disable versioning.

Open the Service template graphical editor:

Open the Palette on the left side of the Canvas Find radon.nodes.azure group. Drag and drop the following node types to the canvas:: AzurePlatform - configures the Azure Python libraries needed AzureStorageAccount - Creates a new Storage account AzureHttpTriggeredFunction - Deploys a new Azure FaaS function. Function code is provided as a zip file. AzureResourceGroup - Creates a new Azure resource group And configure them as described in the following section.

Also, make sure to first activate showing the configuration values of the nodes by clicking on the Properties button on top of the canvas:

NB! Also, remember to change the ResourceGroupName, StorageGroupName, and AzureFunctionName with the names you want to assign for New Azure resources. Avoid using names of resources and functions that already exist, otherwise, it will be hard to test whether we succeed with the deployment later.

Configure the Azure Platform node: Region: northeurope Availiability zone inside the Azure cloud Name: Freely choosable name for this TOSCA node type
Configure the Azure Resource Group node: Resource group name: Resource group to be created NB! Choose a unique name across Azure Region: northeurope Availability zone inside Azure cloud component_version: Not used, can leave empty
Configure the Azure Storage Account node: storage_account_name: Name of the Storage account to be created NB! Choose a unique name across Azure resource_group_name: Name of the resource group (from previous step) region: northeurope Can leave default values: account_type: Standard_RAGRS storage_kind: StorageV2 access_tier: Hot
Configure the Azure HTTP Triggered Function node: storage_account_name: Name of the storage account (from previous step) function_app_name: name of the function to be created NB! Choose a unique name across Azure resource_group_name: Name of the resource group (from previous step) name: Freely choosable name for this TOSCA node type zip_file: /home/ubuntu/function.zip region: northeurope Can leave default values: runtime_version: 3.7 auth_level: anonymous methods: [] functions_version: 2 route: none os_type: Linux route_prefix: api runtime_type: python

NB! Make sure to SAVE the created Service template every once in a while, otherwise, changes might be lost,

To create connections (relationships) between the Azure Node types to each other, first, let's activate the Requirements and Capabilities view by clicking on the respective button on top of the canvas:

Then, we click and drag connections from the Requirements of one node type to the Capabilities of another node type.

Set up the following relationships:

• Define the hosted on relationship to indicate which node should be deployed on which other node:
  1. Drag HostedOn requirement from AzureStorageAccount into AzurePlatform host capability
  2. Drag HostedOn requirement from AzureHttpTriggeredFunction into AzurePlatform host capability
  3. Drag HostedOn requirement from AzureResourceGroup into AzurePlatform host capability
• Define the Depends on type relationship to define which node should be deployed before other nodes are deployed:
  1. Drag DependsOn requirement from AzureHttpTriggeredFunction into AzureStorageAccount feature capability
     • storage account should be deployed before the function
  2. Drag DependsOn requirement from AzureStorageAccount into AzureResourceGroup feature capability
     • resource group should be deployed before the storage account

The result should look something like this:

NB! Check that the relationships you created between nodes in Winery and their direction match this image. Otherwise, you may run into issues later.

Download the created Service template as a .csar file container (basically a zip container) by clicking on the Manage button on the top left get out of the Editor view and choose Export and Download:

NB! Note down the time when you have finished solving this task (14.2). For verification you can check that your service template looks similar to the example image.

Take a screenshot of the finalized Service template in Winery

Exercise 14.3 Setting up RADON xOpera

In this task, we will set up xOpera TOSCA orchestrator, which is able to parse the models we have previously created and deploy them automatically.

opera aims to be a lightweight orchestrator compliant with OASIS TOSCA. XOpera takes the TOSCA service template and manages the life cycle of the nodes inside the service. Life cycle refers to creating, configuring, starting, stopping, and deleting the individual nodes. The life cycle commands of TOSCA node types are implemented as Ansible playbooks, included inside each TOSCA node type.

• Connect to the OpenStack VM over SSH and update the Linux repositories sudo apt update
• Install the python virtual environment sudo apt install -y python3-venv python3-wheel python-wheel-common
• Install Azure CLI package:
  • curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash
• Create a new opera directory: mkdir opera
• Move into the created folder: cd opera
• Create a python virtual environment python3 -m venv .venv
• Activate the virtual environment:
  • source .venv/bin/activate
  • Now you should see (.venv) $ in-front of the command line prompt
• Update pip pip install --upgrade pip
• Install opera inside the virtual environment: pip install opera

NB! Every time you want to execute opera in the following tasks, make sure that you are in the correct folder and have activated the Python Virtual environment.

1. cd opera
2. source .venv/bin/activate
3. opera

Exercise 14.4 Deploying a service template

Now we will use the xOpera orchestrator to deploy the previously created (and saved as a .csar file) TOSCA service template.

NB! Note down the time when you have started solving this task (14.4)

1. Copy the downloaded .csar file into the VM.
   • Just like you have moved files into VMs in the previous practice sessions. (e.g. using scp)
2. Also copy the function code function.zip file into the VM.
   • The file MUST be located inside the home folder of the ubuntu user (/home/ubuntu/function.zip)
   • You can download the zip file containing the function from here: https://owncloud.ut.ee/owncloud/index.php/s/NAX9fBfodTpFf3j
   • You will find the password for the link in the Slack channel practice14-tosca as a pinned message
   • We are using the exact same function as in last week's practice session. The zip file contains the function code from the same repository.
3. Log into the Azure Cloud inside the VM:
   • PS! Unfortunately, because we are using the University of Tartu Azure tenant, we do not have access to Azure API credentials to automate the login step also using xOpera. So we have to manually log in before calling xOpera.
   • az login
   • Follow the steps just like in the last week's practice session
4. Make sure Python Virtual environment is active
   • cd ~/opera
   • source .venv/bin/activate
5. Use xOpera to Deploy the TOSCA CSAR file:
   • opera deploy -r SERVICE.csar
     • Replace SERVICE with the correct name of the file.
     • -r specifies that xOpera should resume any ongoing deployment that might have failed.
     • If you run into any errors, you may have to use opera deploy -c SERVICE.csar instead, if you want to make sure xOpera deploys everything again from a clean state and does not resume from the latest state.
   • If you get any errors, scroll up and try to find the actual error message inside the JSON output of the command.
     • NB! Check the following section for an example where the error can be located.
6. xOpera will install necessary Azure libraries inside the local python Virtual environment and deploy and configure Azure resource group, storage account, and FaaS function.
7. Once the deployment is successful, you can log into Azure Portal and verify that the function and other resources have been deployed correctly.
   • Going to https://FUNCTONNAME.azurewebsites.net/ should show it is deployed
   • https://FUNCTONNAME.azurewebsites.net/api/httpexample shows one of the three APIs inside the function(s)
   • If there are no errors with the opera deployment command, you will see output about deployments of the 4 different components (ResourceGroup, Function, ..) take place.

NB! Note down the time when you have finished solving this task (14.4) and everything works properly.

• Take a screenshot of the xOpera output
• Answer: In which order are the 4 components deployed?
  • Why are they being deployed in this order?

Debugging xOpera

Example of a successful xOpera deployment:

In case of errors xOpera may output two types of errors:

1. Python error, which indicates something went wrong
   • Usually the actual error is not displayed here and you can scroll past the Python errors
2. Error inside the Ansible JSON output - if something is wrong on the Ansible side.
   • Anything related to Azure APIs, wrong configuration, etc., will be shown inside the JSON output. Look at the following picture of how to find such errors

Example of how to find an error inside the xOpera output:

• If you run into any errors, you may have to use opera deploy -c SERVICE.csar instead, if you want to make sure xOpera deploys everything again from scratch.

If you are interested in how the Azure TOSCA node types are defined and how their life cycle commands (create, delete) are implemented using Ansible, then you can check their GitHub repository folders: https://github.com/UT-Cloud-Computing-Course/radon-particles/tree/master/nodetypes/radon.nodes.azure

Deliverables:

1. Upload the CSAR file
2. Link to the function deployed in Azure
3. Take a screenshot of the finalized Service template in Winery
4. Take a screenshot of the xOpera output
5. Answer: How long did it take you to solve task 14.2 ?
6. Answer: How long did it take you to solve task 14.4 ?
7. Answer: In which order are the 4 components deployed at task 14.4 ? Why are they being deployed in this order?
8. Answer: Do you have any suggestions for the authors of Winery or xOpera to make the process of designing and deploying Cloud models more convenient?
9. Delete the OpenStack VM

14. lab 14

Possible solutions to potential issues

1. If you get an error that a Storage account or resource group does not exist:
   1. Check that you have specified their names exactly the same way in all the nodes.
   2. Check that the relationships in Winery match the example image. Otherwise, TOSCA nodes may be deployed in the wrong order and the function is deployed before the storage account exists and deployment will thus fail.
2. How to get xOpera to redeploy node types it previously already deployed.
   • You can use the clean state deployment option to make sure xOpera reconfigures everything again:
   • opera deploy --clean-state shivaservice.csar
3. The storage account named mystorage is already taken.
   • For Azure storage accounts, you must provide a name for the resource that is unique across Azure.
   • Solution is to rename the storage account name in Winery and deploy the service again.
   • Use clean state deployment:
     • opera deploy --clean-state shivaservice.csar
   • Make sure you rename the storage account name also inside the other nodes if they use the storage account name.