Hajussüsteemid - Kursused - Arvutiteaduse instituut

Lectures

Weekly lectures

WED 14:15 Liivi 2 - 405

Lecture slides (pdf); Lecture Notes (pdf)

Lecture 1: Introduction to Distributed Systems
- Sample code used during the lecture:
  - udp_sock_send.py
  - udp_sock_receive.py
Lecture 2: Introduction to networks and UDP communication
- Sample code:
Lecture 3:
- Sample code:
  - Practice_1: UDP Multicasting
    - Multicast allows you to send message only to the interested parties and the message is transmitted only once. In order to receive multicast the receiver must be subscribed to particular multicast group in the network. Multicast group is just a specific IP address ranges (according to IANA). (Note: All codes are tested on the Ubuntu 18.04 LTS system with the Python version 2.7.15rc1)
    - Receiving Multicast: Create the receiver server
      - udp_multicast_receiver.py
    - Sending Multicast: Create a multicast sender that will send a message to a multicast group
      - udp_multicast_sender.py
  - Practice_2: Public message board application
    - Download the following package, which includes board.py, protocol.py, server.py and client.py. First, run the server.py that is waiting for clients’ messages. Then run the client.py with the following arguments:
      - –H : server IP address (if it is the local machine use 127.0.0.1)
      - -p : server port (in our case it is 7777)
      - -m : message to publish
      - For example, $ python client.py –H 192.168.1.233 –p 7777 –m “Hello”
      - Message_Board.zip
  - Practice_3: Public message board application (State-full Protocol, Introducing sessions on UDP)
    msgBserver.zip
    msgBclient.zip
Lecture 4:
- Sample codes:
- 1. Create a TCP socket, bind, and listen it
- 2. TCP socket, listen and accept a connection
  - tcp_sock_listen_and_accept.py
  - tcp_sock_connect.py
- 3. TCP socket, send and receive a small message
  - tcp_sock_receive.py
  - tcp_sock_send.py
- 4. TCP socket, send and receive a big block
  - tcp_sock_bigblock_receive.py
  - tcp_sock_bigblock_send.py
- 5. TCP message board application
  - TCP_mboard_server.zip (run mboard_server.py)
  - TCP_mboard_client.zip (run mboard_client.py)
- 6. TCP server to upload files
Lecture 5:
- Sample codes:
- Threads modules in python:
- Python threads are accessible via thread.py, threading.py, and multiprocessing.py modules. Although thread.py module is very effective for low-level threading, it is very limited compared with the newer threading.py module that gives high-level support for threads.
  - 1. Thread module: The simplest and easy way to use a Thread is to instantiate it with a target function and call start_new_thread() to let it begin working.
    - Simple_thread.py
  - 2. Threading module: It contains all the methods of the thread module and more.
    - Simple_threading.py
    - Using Argument with threads:
    - Simple_threading_ag.py
  - 3. Identifying the current thread:
    - identifying_the_thread.py
  - 4. Logging debug: the logging module supports embedding the thread name in every log message
    - identifying_the_thread_debug.py
  - 5. Daemon threads: Till now all the example we showed the programs waited till all the threads exit and complete their work. However, when a programs spawn a thread as daemon that runs without blocking the main program from exiting.
    - daemon_threading.py
    - In order to wait until a daemon thread has completed its work, we can use the join() method
    - daemon_threading_join.py
  - 6. Locks in threads: It is very important to prevent and control corruption of data when handling any communication. We have to use lock object in our code to watch out the accesses to an object.
    - threading_lock_objects.py
  - 7. Conditional Variables in threads:Typical situation in thread programming is when we need to acquire a lock for a queue and pop one object from the queue. If the queue is empty we have to wait until the queue has at least one object.
    - threading_cond_lock.py
Lecture 6:
- Sample codes:
  - 1.Threads in network applications:
    - HTTP Download manager, single threaded
    - singlethreaded_http_download.py
    - HTTP Download manager multi- threaded
    - multithreaded_http_download.py
  - 2.Client-server architectures
    - Single threaded server, one client at most
      - singlethreaded_server_one_client.py
    - Single threaded server, many clients (still processed sequentially)
      - singlethreaded_server_many_clients.py
    - Clients simulator for servers
      - simulating_many_connections.py
  - 3. Multi threaded server with two improved versions
    - Using delegation pattern (thread-per-socket)
    - multithreaded_server_delegation_pattern.py
    - Using pre-fork
    - multithreaded_server_prefork_pattern.py
  - 4. Single threaded server for heavy payload
    - singlethreaded_server_heavy_request.py
    - Client simulator
    - simulating_many_requests.py
  - 5. Combining thread-per-socket and thread-per-request
    - Several patterns can be combined in one server application, for example the pipeline model can be employed if separate the tasks (receiving, processing, sending), therefore we have a fixed number of workers assigned to groups \receivers", \handlers" and \senders". Each task assumes some input and output, and we use three dierent queues to collect \connections", \requests" and \responses". The worker dependent on his group is responsible for taking input task from one queue and putting a result into second one.
      - multithreaded_server_combined_pipeline.py
    - Another combination could be the fixed number of group managers and on-demand workers. In this case there are four fixed threads (main, request-manager, receiver-manager, sender-manager), each manager can create additional threads for handling the corresponding tasks.
      - multithreaded_server_combined_delegation.py
  - 6. Combining threading and multiprocessing
    - Python's threads are good for handling I/O but for parallelizing CPU intensive task they do not suit well. Python threads will never be scheduled to different CPU cores, and in order to benefit from multi-core CPU the Python multiprocessing module has to be used.
      - multiprocess_server_combined_pipeline.py
  - 7. Object oriented implementation
    - Single threaded server OOP implementation
      - oop_singlethreaded_server_many_clients.py
    - Multi threaded server OOP implementation
      - oop_multithreaded_server_many_clients.py
  - 8. Message board implementations using long-lived TCP connections
    - long_lived_tcp_message_board_server.py
    - long_lived_tcp_message_board_client.py
  - 9. Multi-player implementation of the simple "guess word" game where multiple users guess word by proposing letters one by one. Usually in such game, the players proposes letters one after another and the moderator ensures that they never propose letters simultaneously.
    - shared_state_guessword_game_server.py
    - shared_state_guessword_game_client.py
  - 10. Implement interactive Tic-Tac-Toe game as network multi-player application (https://www.wikihow.com/Play-Tic-Tac-Toe)
    - game.zip
- Sample codes for the Windows OS (Please extract files and open it as a project)
  - Lecture6_Codes_Windows.zip
Lecture 7:
- Sample codes: Remote Procedure Calls (RPC) and Distributed Objects
  - 1.Remote Procedure Calls (RPC)
    - The RPC is equivalent to a function call. This means that when an RPC is made the calling arguments are passed to the remote procedure and the caller waits for a response to be returned from the remote procedure.
    - In our example application, we will use XML-RPC implementation for Python.
      - Simple RPC Message Board Server rpc_mboard_server.py
      - Simple RPC Message Board Client rpc_mboard_client.py
  - 2. Distributed Objects (DO)
    - DO allow objects to be referred from outside of the current application's address spaces. Moreover, the objects can be distributed across the different application running on different machines.
    - In our example application, we will use Pyro, Python Remote Objects framework.
      - Simple Chat server using Pyro pyro_chat_server.py
      - Simple Chat client using Pyro pyro_chat_client.py
  - 3. Simple file transfer application (Pyro based) allowing listing, downloading, uploading, renaming, and deleting files.
    - Simple Pyro Client-Server Application Pyro_Client_Server_App.zip
    - Simple Pyro Client-Server Application (for Windows OS) Windows_Pyro_Client_Server_App.zip
Lecture 8:
- Publish-subscribe systems (Distributed event-based systems)
- In this example we simulate a simple pub/sub system. We need to install a message broker (here it is RabbitMQ) and a Python client Pika.
- For windows:
  - Install Erlang (http://www.erlang.org/downloads)
  - Install RabbitMQ (https://www.rabbitmq.com/install-windows.html)
  - Install pika (C:\Python27\Scripts>pip.exe install pika)
- For Linux:
  - Install Erlang and RabbitMQ Server(https://tecadmin.net/install-rabbitmq-server-on-ubuntu/)(https://www.cloudamqp.com/blog/2015-05-18-part1-rabbitmq-for-beginners-what-is-rabbitmq.html#set_up_instance)
    - receive.py
    - send.py
- Tutorial about the pub/sub system (https://www.rabbitmq.com/tutorials/tutorial-one-python.html)
Lecture 9:
- Tutorial Simple_Message_Queue_Pika_Python_client.pdf
  - Sample codes
- Continuing with tutorials at (https://www.rabbitmq.com/getstarted.html)
Lecture 10: P2P communication
- Simple P2P examples, based on group communication:
  - Example_1: P2P chat application (UDP Multicast) p2p_chat_mc.py
  - Example_2: P2P chat application (UDP Broadcast) p2p_chat_bc.py

Hajussüsteemid 2018/19 sügis

Lectures

Weekly lectures