Hajussüsteemid - Kursused - Arvutiteaduse instituut

Seminar 1: Introduction and Project Setup

Goal: To understand the practical scenario and configuration of the project environment and tools.

Introduction

Welcome to the practice sessions for the Distributed Systems course! In the rapidly evolving landscape of modern technology, the seamless interaction and collaboration of various components are at the core of many innovative solutions. Distributed Systems, a field that explores the intricacies of connecting and coordinating multiple services, plays a pivotal role in shaping the architecture of modern applications.

Imagine a scenario where a user initiates a request from the frontend of an application. As this request traverses through the system, the backend services, ranging from fraud detection to recommendation engines, rules engines, third-party payment providers, auditing mechanisms, and distributed databases, work in tandem to craft a cohesive response for the end-user.

Our journey through this Distributed Systems course will delve into the fascinating world of interconnected components. We will unravel the complexities of orchestrating diverse services, understanding their inner workings, and exploring the algorithms and strategies that drive their seamless cooperation. This year's practice sessions are designed to provide a hands-on experience, allowing you to incrementally build and connect various components. Through practical exercises, you will gain a deeper understanding of the challenges and nuances inherent in designing and implementing distributed systems. So, buckle up for an exciting exploration of distributed systems, where theory meets practice, and you'll be empowered to architect robust and scalable solutions for the challenges of the digital age. Throughout these sessions, you will embark on the implementation of the entire flow, albeit at a scaled-down level.

Along the way, you will grapple with diverse subproblems, including communication, event ordering, leader election, consensus mechanisms, scalability challenges, and monitoring solutions, providing a holistic and practical understanding of the intricacies involved in building robust distributed systems.

In order to contextualize these challenges, we'll introduce a couple of scenarios that mirror real-world complexities, allowing you to apply theoretical concepts to a practical, scenario-based setting.

Scenario

We now embark on a journey through the intricate landscape of an online bookshop. Imagine a digital marketplace, where customers explore a catalogue of books and navigate through the virtual shelves. As our scenario unfolds, during the next practice sessions, we will delve into the challenges and innovations that lie beneath the surface, focusing on the pivotal moment of a customer's checkout request.

In the frontend of our online bookshop, a user initiates a checkout request, setting in motion a sequence of events that echoes through the entire backend system. The process of fulfilling the order begins with a request passing through an orchestrator service, which in turn triggers a set of backend services. These backend services include, among many others, payment processing, inventory management, order fulfilment, recommendation engines, user authentication, etc… – each representing a crucial component in the grand architecture of our online store.

Here lies the kernel of our exploration. You will actively participate in building the intricate connections of these components. Your task is to implement and optimize the entire flow, tackling various subproblems that mirror the complexities faced by real-world systems. Communication between services, ordering of events to ensure a seamless transaction, leader election for critical decision-making, consensus mechanisms to maintain data integrity, scalability to accommodate growing demands, or robust monitoring solutions, these are some of the distributed systems challenges you'll encounter.

Through the lens of our online bookshop scenario, you will gain an understanding of the distributed system topics we've discussed earlier. Your journey will not only be about conceptualizing these challenges, but about actively overcoming them in a hands-on environment, where the simple act of checking out an order at an online bookshop transforms into an exploration of architecture and technology.

Project Setup and Tools

For our Distributed Systems project, we'll be utilizing a well-structured repository with an initial project layout, leveraging powerful tools such as Docker, Docker Compose, and Python.

Python: Our programming language of choice for this project, Python offers simplicity and versatility. With a rich ecosystem of libraries, it is an ideal language for prototyping distributed systems.

Docker: A containerization platform that allows you to package applications and their dependencies into a standardized unit, ensuring consistency across different environments.

Docker Compose: A tool for defining and running multi-container Docker applications. With Docker Compose, you can define a multi-container application in a single file, then spin up your application with a single command.

As we progress through the sessions, additional tools and technologies will be introduced, building upon this foundation to create a robust and scalable distributed system. Ensure that you have Python, Docker and Docker Compose installed and updated on your machine before the first session to make the most of our hands-on practice.

Python Documentation: https://docs.python.org/3/, https://www.python.org/downloads/

Docker Documentation: https://docs.docker.com, https://docs.docker.com/get-docker/

Docker Compose Documentation: https://docs.docker.com/compose/, https://docs.docker.com/compose/install/

Feel free to explore these resources to familiarize yourself with Docker, Docker Compose, and Python as we embark on our Distributed Systems journey.

Task:

Install Essential Tools:
- Ensure that you have Python, Docker, and Docker Compose installed on your machine. Refer to the documentation links provided earlier to guide you through the installation process.
Team Formation:
- Form teams of two (2) students each, with at least one team member possessing coding skills. Effective collaboration will be key to the success of this project.
Explore the Initial Repository:
- Fork the initial project repository available at https://github.com/mobile-cloud-computing/ds-practice-2024 to your personal GitHub account.
- Add your group mate and the designated practice session TA as collaborators to your forked repository.
- Follow the initial instructions provided in the repository to set up your development environment.

These initial steps are crucial to laying the groundwork for our upcoming sessions. Take the time to familiarize yourself with the project structure, ensuring a smooth start to our hands-on exploration of distributed systems. If you encounter any issues or have questions, don't hesitate to reach out to the practice session TA for assistance. Happy coding!

Additional Notes

While we have provided an initial scenario involving an online bookshop for our project, we encourage creativity and innovation. If you and your teammate have a different yet analogous scenario in mind, feel free to explore it! Your chosen scenario should share similarities with the online bookshop order checkout, presenting challenges that align with the distributed system topics we'll cover. For instance, developing a distributed system for an online payments or money transfer platform, or a ride-hailing app, are some of the valid ideas.

Discuss your alternative scenario ideas with the practice session TA for approval and guidance. This opportunity allows you to tailor the project to your interests and provides a more personalized learning experience. Remember, the goal is to apply distributed system concepts in a real-world context, so choose a scenario that excites and challenges you.

Hajussüsteemid 2023/24 kevad