[Dahua cloud native] wonderful relationship between message queue and express cabinet (with video)

I've read a lot about cloud primitives , Or cloud mountain fog cover , Or you're too high and too few . So the author has written 《 Big talk cloud 》 The idea of the series , Expect to use the most popular 、 Simple language 、 A memorable scene to illustrate ： Composition and application relationship in cloud protozoan ecosystem .

In the previous article, we have introduced... In vernacular docker、k8s、 Grayscale Publishing 、 Microservices 、 Load balancing, etc , In this article, let's take a look at the components commonly used in the cloud native environment - Message queue . When it comes to message queues, maybe some friends often listen to others talk about some nouns , such as ： Service program decoupling , Handle flow peak shaving , Improve user experience through asynchronous processing , Buffering batch processing improves processing performance . The author is good at explaining in vernacular , So I don't need professional terms to explain professional problems . I always think the function of message queue is very similar to that of express cabinet , What kind of similarity method ？ Let me tell you more about .

One 、 The message queue is a delivery cabinet

Let's compare the express cabinet with the message queue

• Message queues are compared to express cabinets ： Many manufacturers produce express cabinets , Such as ： Fengchao （apache kafka）, Express mail （alibaba RocketMQ）, Nearest neighbor treasure （ActiveMQ） wait , Anyway, these are the only ones commonly used . The express cabinet is responsible for temporary storage of mail , Message queue is responsible for temporarily saving message data .
• Couriers are compared to message producers ： The courier is responsible for delivering mail to the express cabinet , The producer is responsible for delivering messages to the message queue . It's the same as different ！
• Consumers are compared to news consumers ： Maybe this example is too appropriate , So much so that this sentence is nonsense . Nonsense still needs to be said , Consumers in life pick up e-mail , Consumers in the program get message data .

Two 、 Express cabinet ( Message queue ) Benefits

Let's first review in How do you spend your days without express cabinets ： One morning, I suddenly received a call from the courier ：“ brother , There's your express ”. I thought it was terrible ：“ Whether you come early or late , I'm going to work soon , You come at this time . ok , I'll wait for you ”. As a result, it is possible that the courier is very unreliable , I'll talk about the traffic jam later , I'll be right there , You're late for work . This situation makes you collapse ！

And then one day , A thing called express cabinet suddenly appeared in the community , This thing is good .“ brother , You express it ”, Thinking who your brother is ：“ ah , You put it in the express cabinet , I come back from work in the evening to pick up ”. The courier happily put the express into the express cabinet , Start making the next call , One morning 10 Messages . If everyone is delivered to the door, the courier will take at least half an hour . Ok now ,9 A delivery cabinet ,1 A user asked for door-to-door delivery ,10 In minutes . The courier thinks this thing is really good ！

The courier is happy , Consumers and users are actually very satisfied , Some shopaholics may charge 10 An email . When there is no express cabinet , The courier will pick it up once he calls （ Wait a minute ） Courier . With the express cabinet , I took them all together when I got off work . The above example , Embodies the message queue ( Express cabinet ) Several advantages of , Please comment carefully ：

• Asynchronous decoupling ： With the express cabinet , Consumers don't have to wait for the courier , User experience enhancements . Consumers and producers ( Courier ) Decoupling between , Not because of the other party's operation behavior , Procedures that affect your independence . Users don't have to wait and receive events. Blocking takes time .
• Traffic peak clipping ： Let's assume an extreme case , You bought... Through various channels 1000 This book , Suddenly, I'll call you intensively one hour . You certainly don't have the ability to charge for an hour 1000 The ability to send an email , So you ask the courier to put the mail in the express cabinet . So you can handle it according to your ability , Pick up the mail according to your own schedule . Similarly, our consumer programs are facing multiple users 、 In the case of highly concurrent requests , Putting the data into the message queue for saving can cut the peak of traffic data , Data consumption shall be carried out according to the processing capacity and resources of the program .
• Buffer batch ： Batch delivery by producers , Shuang ！ Consumers get multiple emails at one time , Shuang ！

3、 ... and 、 Disadvantages of introducing express cabinet

Said so many advantages , Is there any defect in the express cabinet ？ Of course.

1. Introduce complexity . without doubt , Express cabinet ( Message queue ) This thing is extra , It doesn't exist in the original collection process . So I need a place to put it , Fire protection is also needed 、 Anti theft and moisture-proof , It needs to be maintained . The same is true for Message Queuing Middleware , You need to install it in the server area , And maintain it .

2. It will cause temporary data inconsistency . If there is no express cabinet , When you receive an email, you really receive . But after using the express cabinet , You received " The message that the mail is put into the express cabinet ", But there will be a certain delay when you really get the email . Of course, you will eventually get the mail , choice " Message queue " Express cabinet , Is to endure temporary inconsistencies , Accept " Final consistency ".

3. Of course, in extreme cases , The express cabinet is broken , You have to inevitably accept " Mail may be lost " Fact , For communities with high security coefficient, this will hardly happen .