Seminar 1: Introduction and Project Setup
Goal: To understand the practical scenario and configuration of the project environment and tools.
Introduction
Welcome to the practice sessions for the Distributed Systems course! Modern applications rely on multiple interconnected services working seamlessly together. Distributed Systems explore how these components coordinate, communicate, and scale efficiently.
Imagine a simple user request in an application—behind the scenes, fraud detection, recommendation engines, payment providers, and databases collaborate to process it. In this course, you'll get hands-on experience in designing, implementing, and optimizing such interactions. You'll tackle key challenges like communication, event ordering, leader election, consensus, scalability, and monitoring, preparing you to build robust and scalable systems.
To make these concepts concrete, we'll work through a real-world scenario.
Scenario: The Online Bookshop
Imagine you’re building an online bookshop, where customers browse, select books, and place orders. While this seems straightforward, behind the scenes, a simple checkout request triggers a complex chain of interactions between multiple distributed services. When a customer clicks to “Buy”, a sequence of backend operations begins, requiring coordination across various independent services to complete the transaction smoothly, for example:
- Orchestrator Service: Coordinates the entire process, ensuring actions execute in the correct order.
- Payment Processing: Validates and processes payments, handling different providers and potential failures.
- Inventory Management: Checks stock availability, reserves books, and updates records.
- Order Fulfillment: Manages shipping logistics, possibly interacting with third-party couriers.
- User Authentication: Verifies the customer's identity and purchase history.
- Recommendation Engine: Updates user preferences and suggests similar books.
- Auditing & Logging: Records transactions for compliance and debugging.
Each of these services runs independently, possibly in different locations, requiring efficient communication, synchronization, and fault tolerance to ensure a seamless user experience. These components must work together efficiently to prevent issues like over-selling inventory, failed payments, or unfulfilled orders.
The Challenges You'll Tackle
Throughout the practice sessions, you will implement and optimize this system while addressing real-world distributed system challenges, including:
- Communication & Event Flow: Ensuring services exchange information reliably.
- Event Ordering: Maintaining proper sequence of actions (e.g., payment before inventory updates).
- Leader Election & Consensus: Handling decision-making across distributed nodes (e.g., inventory updates in multiple databases).
- Scalability & Fault Tolerance: Ensuring the system handles high traffic and recovers from failures.
- Monitoring & Debugging: Tracking service health, identifying bottlenecks, and logging transactions effectively.
By overcoming these challenges, you will develop a deeper understanding of how distributed systems function in real-world applications, and you will apply them in a practical setting, progressing from a simple checkout request to a highly orchestrated backend.
Project Setup and Tools
For our Distributed Systems project, we'll be utilizing a structured repository with an initial project layout, leveraging tools such as Docker, Docker Compose, and Python.
Python: Our programming language of choice for this project, Python offers simplicity and versatility. With a rich ecosystem of libraries, it is an ideal language for prototyping distributed systems.
Docker: A containerization platform that allows you to package applications and their dependencies into a standardized unit, ensuring consistency across different environments.
Docker Compose: A tool for defining and running multi-container Docker applications. With Docker Compose, you can define a multi-container application in a single file, then spin up your application with a single command.
As we progress through the sessions, additional tools and technologies will be introduced, building upon this foundation to create a bigger distributed system. Ensure that you have Python, Docker and Docker Compose installed and updated on your machine before the first session to make the most of our hands-on practice.
Python Documentation: https://docs.python.org/3/, https://www.python.org/downloads/
Docker Documentation: https://docs.docker.com, https://docs.docker.com/get-docker/
Docker Compose Documentation: https://docs.docker.com/compose/, https://docs.docker.com/compose/install/
Feel free to explore these resources to familiarize yourself with Docker, Docker Compose, and Python as we embark on our Distributed Systems journey.
Task:
- Install Essential Tools:
- Ensure that you have Python, Docker, and Docker Compose installed on your machine. Refer to the documentation links provided earlier to guide you through the installation process.
- Team Formation:
- Form teams of two (2) students each, with at least one team member possessing coding skills. Effective collaboration will be key to the success of this project.
- Explore the Initial Repository:
- Fork the initial project repository available at https://github.com/mobile-cloud-computing/ds-practice-2025 to your personal GitHub account.
- Add your group mate and the designated practice session TA as collaborators to your forked repository (TA Eduardo github handler: edurbrito-cyber)
- Follow the initial instructions provided in the repository to set up your development environment.
These initial steps are crucial to laying the groundwork for our upcoming sessions. Take the time to familiarize yourself with the project structure, ensuring a smooth start to our hands-on exploration of distributed systems. If you encounter any issues or have questions, don't hesitate to reach out to the practice session TA for assistance. Happy coding!