The reliability of the message

Production and consumption process

We often use message queues , Here we briefly introduce the reliability of messages , The author takes kafka Message queue is the background . Let's take a look at the whole process of news from generation to consumption ：
Here's the picture ：
The producer sends a message to Broker,Broker After receiving the message , Store it in memory , Persist to cluster storage （ Confirm according to the reliability , It can be adjusted ： For example, if there is no need to ensure that messages are not lost , After receiving the message , Return confirmation without persistence ）, Send confirmation to Producer, Send complete .
consumer , It adopts the method of pulling , After pulling , Consumption is （ Generally, it is the business logic of processing consumption ）, Send confirmation to Broker.

detailed , The author combed the guide map ：

Duplicate message

Through the reliability of the message, we know , In the realization of reliability , The network jitter 、 The system handles busy and other unstable situations
In the sending phase ： Send confirmation timeout , A message resend occurs .
Consumption stage ： Over time consumption , Send a message and resend ; Consumption confirmation timeout , There will also be message resending .
here , We need to respond to repeated messages , Usually on the consumer side , Doing idempotent processing is to do idempotent design for the business logic of consumption messages .
The general business idempotency design is shown in the following diagram ：