Session 7.2: Microservices integration patterns - Event Driven Architecture - Kafka
1. Clone the following repository
$ git clone https://github.com/M-Gharib/ESI-W7.2.git
Note: if you want to create a new Spring Boot project from scratch, you need to install the following dependencies for both the Product and Payment services:
- Spring Web;
- Spring for Apache Kafka;
- Spring Data JPA SQL;
- PostgresSQL Driver SQL;
- Spring Reactive Web;
- Lombok.
A simplified representation of the application is shown in the following diagram.
On receiving a product creation request, orderservice create an order setting the Order status to "Created" and the Payment status to "Pending", and save it within the database. Then, pushes an "OrderDto" to the "orderCreatedTopic" topic.
paymentservice listens to the "orderCreatedTopic" topic, and pulls the "OrderDto" from the "orderCreatedTopic" topic. Then, it fetches the user's balance from the database and checks whether the user's balance is sufficient for paying for the order. If the balance is sufficient, the order cost is subtracted from the balance (update balance), the Payment status is changed to "Completed", and the Order status is also changed to "Completed". If the balance is not sufficient, the Payment status is changed to "Failed", and the Order status is also changed to "Canceled". In both cases, the paymentservice pushes an "OrderDto" with the outcome to the "paymentTopic" topic.
paymentservice listens to the "paymentTopic" topic, and pulls the "OrderDto" from the "orderCreatedTopic" topic. Then, it updates the order information in the database.
A simplified representation of application
The orderservice has the following structure.
orderservice
└── configuration
└── KafkaTopicConfiguration.java
└── controller
└── OrderController.java
└── dto
└── OrderDto.java
└── model
└── Order.java
└── OrderStatus.java
└── PaymentStatus.java
└── repository
└── OrderRepository.java
└── service
└── OrderService.java
2. Check the code in KafkaTopicConfiguration.java, we are creating two topics orderCreatedTopic and paymentTopic. The first is used for pushing OrderDto objects on the creation of an order, and the second is to pull OrderDto objects produced by the payment service.
3. Check the code in Order.java, understand the data model, and how the Order status and the Payment status are incorporated into it.
4. Check the code of the addOrder(OrderDto orderDto) function in OrderService.java, and try to understand how we are mapping the OrderDto to the Order Object, then, how we are setting up the initial status (on creation) of the order and payment, i.e., the order status is set to Created, and the payment status is set to Pending. Then, we are setting the same statuses to the order and payment of the OrderDto before sending it to the orderCreatedTopic topic.
5. Check the code of the updatePaymentinfo(OrderDto orderDto) function in OrderService.java that is listening to the paymentTopic topic. In short, this function will update the order in the database when pulling an OrderDto that is pushed by the paymentservice to the @paymentTopic@@ topic.
6. Check the application.properties in orderservice it contains the producer and consumer configuration for serializing/deserializing OrderDto objects.
The paymentservice has the following structure.
paymentservice
└── dto
└── Order.java
└── OrderStatus.java
└── PaymentStatus.java
└── model
└── UserBalance.java
└── repository
└── PaymentRepository.java
└── service
└── PaymentService.java
7. Check the code in UserBalance.java to understand the data model.
8. Check the code of the processpayment(OrderDto orderDto) function in PaymentService.java that is listening to the orderCreatedTopic topic. In short, this function fetches the user's balance from the database and checks whether the user's balance is sufficient for paying for the order. If the balance is sufficient, the order cost is subtracted from the balance (update balance), the Payment status is changed to "Completed", and the Order status is also changed to "Completed". If the balance is not sufficient, the Payment status is changed to "Failed", and the Order status is also changed to "Canceled". In both cases, the paymentservice pushes an "OrderDto" with the outcome to the "paymentTopic" topic.
9. Check the application.properties in paymentservice it also contains the producer and consumer configuration for serializing/deserializing OrderDto objects.
Running the application
1. Run Docker desktop.
2. Run the following command to start Zookeeper and Kafka broker.
$ docker compose up -d
3. Run orderservice, then, open Postgres (pgAdmin) and check the created database (orderservice_db) and the created table (orderstable)
4. Run paymentservice, then, check the created database in Postgres (pgAdmin (paymentservice_db), the created table (userbalancetable), and the inserted records.
5. Open a new terminal and launch the kafka-console-consumer and make it listen to the orderCreatedTopic topic, by typing the following command:
$ docker exec --interactive --tty broker kafka-console-consumer --bootstrap-server broker:9092 --topic orderCreatedTopic --from-beginning
Now, any message written to the orderCreatedTopic topic will appear in this terminal.
6. Open a new terminal and launch the paymentTopic and make it listen to the orderCreatedTopic topic, by typing the following command:
$ docker exec --interactive --tty broker kafka-console-consumer --bootstrap-server broker:9092 --topic paymentTopic --from-beginning
Similarly, any message written to the paymentTopic topic will appear in this terminal.
7. Send the following request in RestClientFile.rest, then, check the console of both services, you should see that the Order object has been sent from the payment service and received by the payment service. Also, check two terminals that run kafka-console-consumer and listen to the orderCreatedTopic and paymentTopic respectively, where you should see the objects that have been sent to both topics.
POST http://localhost:8082/api/orders HTTP/1.1
content-type: application/json
{
"id": 1,
"userId": 1,
"productId": 101,
"price": 200
}
8. Check the content of both userbalancetable and orderstable tables, you should see the results have been already saved to the databases.
9. Send the following request through RestClientFile.rest, where the use balance cannot cover, i.e., the payment will fail and the order should be canceled.
POST http://localhost:8082/api/orders HTTP/1.1
content-type: application/json
{
"id": 2,
"userId": 1,
"productId": 101,
"price": 2000
}
10. Check the two terminals that run kafka-console-consumer and listen to the orderCreatedTopic and paymentTopic respectively, where you should see the objects that have been sent to both topics. Also, check the content of both userbalancetable and orderstable tables for this order.