Seminar 7: Process/Node replication in python
Goal: To study and practice replication in distributed systems.
Background: There are two primary reasons for replication: reliability and performance.
- Enhancing reliability
- If one replica/node crashes, the system can continue working by switching to other replicas/nodes.
- Protection against hardware crashes and corruption of data
- Improving performance
- Scaling in terms of numbers and geographical area
- Caching is a special form of replication for speeding up processing
- Trade-off between performance and scalability
Besides, replication can help in load sharing where loads on a server are shared among different replicas. It can also enhance the availability of the data. If the replicas are created and data is stored near to the consumers, it would be easier and faster to fetch data.
Replication:
- Copy of the same data in multiple nodes
- Databases (Apache Casaandra), file systems (NFS – Network file sharing)…
- A node that has a copy of the data is called replica
- If some replicas are faulty, others are still accessible
- Spread load across replicas
- If data does not change, data distribution becomes relatively easy
Two types of Replication:
- Active Replication
- The request of the client goes to all the replicas.
- Every replica receives the request in the same order otherwise the system will get inconsistent.
- No need for coordination because each copy processes the same request in the same sequence.
- All replicas respond to the client’s request.
- Pros: The implementation is simple and the codes are the same throughout. Even if a node fails, it will be easily handled by replicas of that node.
- Cons: It increases resource consumption and time complexity. The greater the number of replicas, the greater the memory needed. If some change is done on one replica it should also be done in all others.
- Passive Replication
- The client request goes to the primary/main replica.
- Other replicas act as backup for the primary/main replica.
- Primary replica informs all other backup replicas about any modification done.
- The response is returned to the client by a primary/main replica.
- In case of failure of a primary replica, a backup replica becomes the primary replica.
- Pros: The resource consumption is less as backup servers only come into play when the primary server fails. The time complexity of this is also less as there’s no need for the whole updating in all the nodes replicas.
- Cons: If some failure occurs, the response time is delayed.
Task: From previous seminars, we have already implemented some functions with multi-threads by using RPyC ThreadedServer in python. For this task, we will implement the Passive Replication:
- All the messaging passing is just between the client and the main thread(main replica) in the server.
- Like seminar 6, when the client sends one command/message, the server execute the related function.
- Create N (N>=1) threads in the server side ($ program N).
- These N threads are like other N replicas which just copy the data of the main replica.
- Suggestion: you can create the thread using class in Python (thread class).
- Show the data of all replicas ($ list)
- update the data of the main replica ($ update data)
- The data can be an integer, a string or others.
- set the update(cache) time t ($ cache t).
- When the main replica update its data, other replicas update its own data after t seconds (e.g., t=20).
- Please note that the update process is always running automatically, this command just changes the update time. If the client does not send this command, the default update time is 10 seconds.
- The result should be similar to the screenshots.
Deliverables: please contain the source python file, and also the screenshot of results in terminals. Then put all the file in a single zip.
- If you have different solutions for the tasks, please submit all your solutions with a simple description
Link:
- https://en.wikipedia.org/wiki/Replication_
- https://www.geeksforgeeks.org/what-is-replication-in-distributed-system/
NOTE: Please watch the video or ask us through Slack if something is not clear for you. We usually don't change the points after the deadline.