Practice 11 - Introduction Aneka Cloud

In this Practice session you will work with Aneka cloud computing tool and specially work with Aneka API's. You will run the examples on image processing and BLAST tool for Bag of Task processing on cloud. You also know about hybrid cloud implemented in Aneka cloud(Video Attached).

NB! Aneka clusters are no longer running. You can no longer fully complete the lab.

Practical session communication!

Lab supervisors will provide support through Slack

• We have set up a Slack workspace for lab and course related discussions. Invitation link was sent separately to all students through email.
  • Use the #practice11-aneka Slack channel for lab related questions and discussion.
• When asking questions or support from lab assistant, please make sure to also provide all needed information, including screenshots, configurations, and even code if needed.
  • If code needs to be shown to the lab supervisor, send it (or a link to it) through Slack Direct Messages.

References

Aneka hybrid cloud :https://www.manjrasoft.com/download/2.0/AnekaDynamicProvisioning.pdf
Aneka multi cloud support: https://arxiv.org/ftp/arxiv/papers/1511/1511.08857.pdf
Aneka Thread API:http://www.manjrasoft.com/download/ThreadModel.pdf
Aneka Task API:http://www.manjrasoft.com/download/TaskModel.pdf
Aneka Map Reduce API:http://www.manjrasoft.com/download/MapReduceModel.pdf
Aneka research papers:http://www.manjrasoft.com/manjrasoft_aboutus_research.html

Aneka architecture

Aneka is a PaaS cloud software that facilities the development and deployment of applications with underline support of .net framework. It contains three layers which are infrastructure, middleware and application development which allows the end user to change underline infrastructure and middleware to support rapid and customized development of cloud applications. The infrastructure of Aneka can be multicore machine, grid environment, cluster of machines, private cloud and public cloud. Public cloud is only required when scheduler decides that desired QoS cannot be achieved from other available resources. Middleware provides the support for programming models such as thread, task and MapReduce. It also provides the billing, accounting, resource reservation, hardware profiling and other services. These service in Aneka can be plugged and unplugged as and when required from the Management console provided in the application development and management layer. Aneka also contains many components which are discussed below:

• Aneka Master Container: Aneka master is mainly responsible for the scheduling and monitoring of application tasks and resources in the Aneka. Aneka master also manages the billing and reporting services of all the worker nodes attached with an Aneka master. The end user’s application sends the task to Aneka master which based on the availability of worker nodes further schedules these tasks. If any worker nodes fail without executing the task, master reschedules the task o another available worker node.
• Aneka Worker Container: Aneka worker is the component which basically deals with the execution of the tasks. It contains the executor for each type of programming models available with the Aneka. After completing any assigned task, it sends the result back to master for compiling and starts waiting for new tasks. Worker node is operating system independent, it can be run on Linux or Windows machines.
• Aneka Daemon: Aneka daemons are the basic services which need to be installed in all the machines before Aneka master and worker can be installed.
• Management Studio:Management studio is the interactive interface which end user uses to create and manage cloud environments created inside the Aneka cloud. The end user can easily add private and public cloud resources, manages all added resources, create bills, monitor added resources, add file repositories, and check current statistics of the Aneka cloud.
• Aneka SDK: Aneka also provides SDK for the development of applications which can be directly deployed on the Aneka cloud created using management studio. These SDKs contains programming language specific libraries and other tools.
• Hardware and software requirements to install Aneka Cloud
  • Hardware requirements: Minimum 2GB RAM, 4 Core CPU, 50GB storage.
  • Software requirements: Windows 10 or higher version of OS, .net framework version-3.5 or higher.

Aneka Hybrid Cloud using Dynamic provisioning property

Dynamic provisioning is the ability to dynamically acquire resources and integrate them into existing infrastructures and software systems. In the most common case, resources are Virtual Machines (VMs) acquired from an Infrastructure-as-a-Service (IaaS) cloud provider. Dynamic provisioning in Aneka happens as part of the Fabric Services by offering provisioning services for allocating virtual nodes from public cloud providers to complement local resources. This is mainly achieved as a result of the interaction between two services: the Scheduling Service and the Resource Provisioning Service. The former triggers on-demand provisioning requests based on the system status and the requirements of applications, while the latter is responsible for interacting with IaaS providers to instantiate VMs and deploy Aneka containers to meet the requests. Execution of applications in Aneka happens through allocating tasks to the available set of resources in a dynamic fashion using the existing scheduling algorithms. Scheduling algorithms might be designed to leverage dynamic provisioning to cope with the application or system requirements. The scheduling algorithm makes decisions regarding when and how many resource allocations must take place to meet the application QoS requirements.

Aneka supports interactions with different resource providers, e.g., Amazon Elastic Computer Cloud (EC2), Microsoft Azure, XenServer, and GoGrid, using its dedicated provider-specific resource pool component. The main operations performed by this component are the translation of provisioning requests into provider specific requests, controlling the life cycle of VMs, and shutting them down when they are no longer needed. The life cycle of resource pools and redirecting provisioning requests, their release, or directing queries to the appropriate pool is the responsibility of the pool manager component. The pool manager also notifies the provisioning service when a dynamic resource is activated and terminated

To implement the hybrid cloud using dynamic provisioning, we require two cloud environments. Due to resource constraints and online conduction we can show the deployment in the video attached(In the video, AWS is used as a aneka deployment environment).
/>

Exercise 11.1. Get aquentied with aneka cloud configuration in OpenStack cloud

Aneka can be deployed in many type of resource pools such as public clouds, private clouds, virtualization pool of servers and desktop PC clusters.
System administrator can configure, monitor and manage the Aneka cloud using user interface called Aneka Management Studio as shown in the below figure.

In this practical session we have already configured using OpenStack, So no need to configure. The configured environment is shown below in the figure.You can refer to the link(http://www.manjrasoft.com/manjrasoft_downloads.html) to download and install for just to try but not mandatory in this practice session.

• The red color blocks represents the aneka daemon provides run time environment for containers to run and monitor life cycle of the containers. All daemons from worker nodes send information to master daemon to maintain global catalog of all containers in association with master container.
• The internal architecture of Aneka consists of Master and Work containers. Master container can schedule/distribute the jobs submitted by the user. The purple color blocks represents the Master container acts as a load balancer and scale the worker containers as required and its connected to different resource pools as explained in hybrid cloud scenario. The worker nodes receives the jobs from master and execute & send back the results to master.
  • Master container address is tcp://172.17.66.132:9090 and it acts as an end point to the user.

There are three VM's in the Aneka cloud cluster and configured as follows:

• Master node: 172.17.66.132 and running at tcp port number 9090.
• Worker Node-1: 172.17.65.12 and 02 worker containers running at port numbers 9090,9091.
• Worker Node-2: 172.17.66.60 and 02 worker containers running at port numbers 9090,9091.

NB!! Prerequisites to perform further exercises.

For Windows OS users

• Ping an aneka master virtual machine ping 172.17.66.132. Make sure that your connected to VPN and Disable the firewall.
• Download Visual Studio Community Edition and install( https://visualstudio.microsoft.com/downloads/)

For Mac or Linux users

• Create a Openstack Virtual Machine with following settings:
  • Select Boot Source-->Instance Snapshot
  • Volume Size-->40GB
  • Flavor: m2.xsmall (4 vCPU, 16 GB)
  • Select Available images-->Windows-Aneka
  • Security Groupshiva (do not remove default security group, both should be added)
• Connect the VM using xrdp or any other rdp clients
  • Username Administator
  • Passwordshivu!123
• Visual Studio 2019 Community Edition is already configured in the VM.
  • If you have any issues with Visual Studio configuration, you can reinstall or modify it using the installation file on the Desktop.

Exercise 11.2.Running an image processing application on Aneka cloud

As mentioned earlier applications can be developed and submitted using aneka API's. There are three types of API's Task, Thread and Map Reduce API. You can refer the references section to understand bit better. In this experiment we using inbuilt application called Convolution which uses the Aneka Task API.

• Convolution is an imaging application that performs image filtering by taking advantage of the computing power provided by Aneka.
• More precisely, Convolution allows users to run any kind of convolution filter in a distributed manner by creating multiple filtering tasks which operate on different rectangular portion of the image. The application also allows users to run the filters locally either sequentially or in parallel by recording the timing statistics.
• Users can define their custom convolution filters or run predefined filters.//
  • Step 1: You can download the Convolution application form link Image Convolution and unzip it. It contains the files as shown in the figure below.

The application is composed by different projects:

• ConvolutionDemo: is Windows Form project containing the GUI of the application and the configuration classes.
• ConvolutionFilter: is a Class Library project containing the wrapper classes that allow running any convolution filter on Aneka.
• Imaging: is a Class Library project containing the definition of the object model used to define and implement filter along with some helper classes.
• PictureBox: is a Control Library project containing the PictureBox control used in the application to show the original image and the result of the filtering process.

• Step 2: Open convdemo.exe from the above mentioned list of files and upload an input image File-->Open-->Select image of your own. Don't change any parameters on convolution application.

• Step 3: Now set Aneka master endpoint to the application convolution-->Settings-->Aneka and enter the endpoint address as shown below in figure.
  • Aneka Master IP: 172.17.66.132 and port no:9090
  • Username: Administrator Password: shivu!123

• Step 4: Now set the image processing algorithm you want apply on the image Convolution-->Emboss. Once you set the algorithm name, the application starts running and jobs are submitted to Aneka Cloud.

After few seconds you should see the ouput as shown in the below figure and click on the arrow- it shows the list of workers running your job.

NB!! Take the screenshot here

• Step 5: Let us understand the process how jobs are running.

The whole image is divide into number of slices with slice dimension 112*112 pixels. The collection of slices considered as jobs submitted to the aneka master for processing with an emboss algorithm. The aneka master will receive the jobs and distribute to workers for processing. The collected output from workers send back to master. The master collectively sends back the output image to user.

You can read the for detail understanding of convolution filter Convolution in Aneka. The ConvolutionFilter project contains the classes which deal with Aneka and allow filter parallelization. In particular the following classes are relevant:

• AnekaConvolutionFilter: it inherits the RBConvolutionFilter class which provides already the facilities for splitting the images into overlapping rectangles and fire events when the filtering operation on one rectangles has completed.
• Worker: this class implements the ITask interface and IMatrixFilter and wraps the execution of the filter on a remote node.

Exercise 11.3 Working with Aneka Thread API

The task is to utilize the Aneka Thread API and to print 20 Aneka distributed threads with output as submission time, completion and address of the node where the threads are executed.

The following are required to perform this task:

• Visual studio with .net framework 3.5(Download and install community edition:( https://visualstudio.microsoft.com/downloads/)
  • Choose .Net desktop developement (C#) when asked to select between different Workloads
• Aneka Software development kit(Aneka SDK) (Download link https://owncloud.ut.ee/owncloud/index.php/s/pPXZrP7ii4CXY6q)
• Aneka configuration file(Download link https://owncloud.ut.ee/owncloud/index.php/s/jsWYibKqMqbzyM4)
• Step 1: Creating a C# Console Application in .NET 3.5
  • Open Visual Studio 2019 Click --> Create New Project
  • From New Project Dialog select C#, windows, Console and Click and Select Console App(.Net Framework) and Click Next
  • Provide Project Name-->SimpleThread, and Select .NET Framework-->3.5 Click Next
• Step 2:
  • Add References to the project from the following folders under Solution Explorer
    • Right Click on the references
      • Click on Add Reference
      • Click on Browse Tab
      • Select the following folders one after the other and add them
        • PATH-TO-ANEKA-SDK\Common\ Select all .dll's
        • PATH-TO-ANEKA-SDK\Runtime\Select all .dll's
        • PATH-TO-ANEKA-SDK\Thread Model\Select all .dll's
• Step 3: Letus modify the Program.cs to print the required solution.

  • Double click on Program.cs in Solution explorer.
  • Let us add the header file to include Aneka Thread API in the program, copy and paste the following snippet

using System;
using Aneka;
using Aneka.Threading;
using Aneka.Entity;

• create a method to print "Hello World with your Name". Modify it to print out your name.

     [Serializable]
        public class HelloWorld
        {
            public string result;
            public int i;

            public void HelloPrint()
            {
                Console.WriteLine("Inside HelloPrint");
                result = "Hello World Your Name";

            }

        }

• Now in the Main() function, let us create a Aneka Thread Application using AnekaApplication<AnekaThread, ThreadManager> app = null;
• Let us add aneka configuration file by creating configuration object Configuration conf = Configuration.GetConfiguration(@"Path-to-Conf\conf.xml");, please add path of your downloaded conf.xml. This will contains the end point of Master(SchedulerUri value="tcp://172.17.66.132:9090/Aneka"), Let's see:

<?xml version="1.0" encoding="utf-8" ?>
<Aneka>
  <UseFileTransfer value="true" />
  <Workspace value="." />
  <SingleSubmission value="false" />
  <ResubmitMode value="AUTO" />
  <PollingTime value="1000" />
  <LogMessages value="true" />
  <SchedulerUri value="tcp://172.17.66.132:9090/Aneka" />
  <UserCredential type="Aneka.Security.UserCredentials" assembly="Aneka">
    <Instance username="Administrator" password="shivu!123"/>
  </UserCredential>
</Aneka>

• Now let's create an object of HelloWorld method HelloWorld hw = new HelloWorld();
• Create an array of 5 Aneka Threads: AnekaThread[] th = new AnekaThread[5];
• Now start the threads to call the HelloPrint() method frome Hello World Class

  for (int i = 0; i < 5; i++)
                  {
                      th[i] = new AnekaThread(hw.HelloPrint, app);
                      th[i].Start();
  
                  }

• Now join the output of the threads and print the Node Id, Submission Time and Completion Time

 for (int i = 0; i < 5; i++)
                {
                    th[i].Join();
                    hw = (HelloWorld)th[i].Target;
                    Console.WriteLine("Value : {0} , NodeId:{1},Submission Time:{2},Completion Time{3}", hw.result, th[i].NodeId, th[i].SubmissionTime, th[i].CompletionTime);


            }

• The overall code structure of Program.cs should look like this:

using System;
using System.Collections.Generic;
using System.Text;
using System.Threading;
using Aneka;
using Aneka.Threading;
using Aneka.Entity;

namespace SimpleThread
{
    class Program
{
    [Serializable]
    public class HelloWorld
    {
        public string result;
        public int i;

        public void HelloPrint()
        {
            Console.WriteLine("Inside HelloPrint");
            result = "Hello World Your Name";

        }

    }

    static void Main(string[] args)
    {

        AnekaApplication<AnekaThread, ThreadManager> app = null;
            try
            {
                Logger.Start();

                Configuration conf = Configuration.GetConfiguration(@"C:\Users\poojara\Desktop\conf.xml");
                app = new AnekaApplication<AnekaThread, ThreadManager>(conf);
                HelloWorld hw = new HelloWorld();
                AnekaThread[] th = new AnekaThread[5];
                for (int i = 0; i < 5; i++)
                {
                    th[i] = new AnekaThread(hw.HelloPrint, app);
                    th[i].Start();

                }
                for (int i = 0; i < 5; i++)
                {
                    th[i].Join();
                    hw = (HelloWorld)th[i].Target;
                    Console.WriteLine("Value : {0} , NodeId:{1},Submission Time:{2},Completion Time{3}", hw.result, th[i].NodeId, th[i].SubmissionTime, th[i].CompletionTime);
                }
            }
            finally
            {

                Logger.Stop();
                app.StopExecution();

            }
    }
}
}

• Try to modify the logic to run for 50 threads.
  • Make sure that you have made the following changes
  • Check the path of conf.xml
  • Check that you added the correct References from Aneka SDK folder as mentioned
  • Check for modification of your name in HelloPrint() method.
• Build the project Click Build--> Build Solution.
• To run the project Press Ctrl+F5 and you should see the output as mentioned in the figure below.

NB!! Take the screenshot of console with output.

PS! If the console window closes before you can make the screenshot, then make sure you use the Ctrl+F5 shortcut to run the program.

Bonus exercise

Bonus task is to utilize the Aneka Task API to display the host name & IP address of the Worker and Application Id, Submission Time, Completion time, Schedule Time.

• Download the the complete source and project here AnekaTaskDemo
• Unzip the project and open AnekaTaskDemo folder
• Right click on AnekaTaskDemo.csproj and open with Visual studio 2019
• Check the path of conf.xml in the MyTaskDemo.cs program and modify accordingly.
• Class Test has logic to get hostname and IP address
• You can refer the manual for understanding task programming Aneka Task API Manual
• In the program @ line number 78 will display the submission time.
• In the same way,try to display the Application Id,Submission Time,Completion time, Schedule Time.
• Run the project Ctrl+F5.
• Console will display the output as

NB!! Take the screenshot of console with output.

Deliverables:

1. Program.cs code
2. Screenshot of the console with required output

Deliverables:

• From 11.1: Answer the questions:
  1. What is the functionality of Aneka Master and Aneka Deamon?
  2. How is the Hybrid Cloud model supported in Aneka?
• From 11.2: Screenshot of convolution application with output of the image
• From 11.3: Screenshot of the console with required output and Program.cs code

11. lab 11