Hajussüsteemid 2023/24 kevad

Seminar 5: Ordering of Events

Goal: Explore the crucial aspect of event ordering, laying the groundwork for a seamless and synchronized flow of transactions.

Introduction

In Session 5 of our Distributed Systems course, we delve into the concept of event ordering and its significance within the context of our online bookshop's checkout process. Event ordering plays a pivotal role in ensuring the integrity and consistency of transaction flows, especially in distributed systems where multiple events occur across multiple services. By meticulously ordering events, we establish a coherent sequence of operations, enabling accurate tracking and synchronization of transactional activities. In this session, we will leverage vector clocks as a robust mechanism to track the order of events across various services.

Some details of the checkout process can be properly ordered to ensure logical execution of the entire flow, to reduce the consumption of unnecessary resources, or achieve correctness and validation of processes. For instance, it doesn't make sense that the fraud-detection service verifies if the request is fraudulent (an expensive operation), if there are mandatory fields in the request which were not filled by the user. This should first be cleared by the transaction-verification service and, only then, the fraud-detection service should execute its workload.

Task

Modify and extend your implementation to achieve the following functionality:

• OrderID: The orchestrator should generate and send a unique OrderID when dispatching the order data to the rest of the backend services.
• Backend services: Now, you should alter the order of execution of an order request. Before, each service had its functionality called by the orchestrator, executed the respective function independently, and delivered its response back to the orchestrator. You should now establish some relations between intermediate events:
  • Initialization: The orchestrator sends the data, via its worker threads, to each service. Now, these services won’t execute their operations immediately, but rather cache or hold the order data temporarily. Along with holding it, each of the 3 backend services will also initialize a vector clock for every order. One suggestion is that some of the threads created by the orchestrator may finish right away, and only one thread may wait for the end of the execution flow (see next item). Feel free to structure the logic as you see fit.
  • Event ordering: Establishing causal relations between the events, you should order them to achieve a meaningful execution flow. One example:
    (event) a. transaction-verification service verifies if the order items (books) are not an empty list.
    b. transaction-verification service verifies if the mandatory user data (name, contact, address…) is all filled in.
    c. transaction-verification service verifies if the credit card information is in the correct format.
    d. fraud-detection service checks the user data for fraud.
    e. fraud-detection service checks the credit card data for fraud.
    f. suggestions service generates book suggestions.
    Relation between these events: a -> b , b -> d , d -> c , c -> e , e -> f
    Establish a similar order of events in any logic you see meaningful in your current project, with a minimum of 5 events, and proceed with the implementation. No extra points will be awarded for more events, the goal is to achieve and show a correct implementation of vector clocks, so don’t spend much time adding extra events. Also, the inner functionality of each event is not very relevant - please, use the same or similar dummy logic you had in the first checkpoint, to simulate or execute each event task.
  • New gRPC functions: Modify your previous gRPC protocols and implement the extra gRPC functions for each event, in the various backend services, to trigger the new functionalities.
    • Vector clocks: Along with it, the vector clocks of each orderID in each service should be updated and propagated correctly. Upon calling each event in each service, log the current vector clock for the given orderID.
    • Response handling: Devise the response mechanism. Now, if failure was detected in any intermediate event, it should be immediately propagated back to the orchestrator. Otherwise, if no failure was detected throughout the entire flow of events, the list of recommended books should be propagated to the orchestrator. In any of the cases, the orchestrator should then stop all the worker threads and answer back to the user.
    • Broadcast: For bonus points, implement a way, in the orchestrator, to broadcast to the other services an instruction to clear the stored order data. This should happen during or after the termination of the worker threads, in either case of failure or success. This event should be the final event in the flow - the orchestrator should attach to its broadcast message the final vector clock VC_f. If the other services have a local VC <= VC_f, the flow is considered correct and the services can safely clear the data, otherwise an error should be reported.