MQ Understanding ：

If product selection

Kafka

advantage ： The throughput is very large , Very good performance , Cluster high availability .

shortcoming ： Lose data , The function is relatively simple

Use scenarios ： Log analysis 、 Big data acquisition
RebbitMQ

advantage ： High message reliability , Comprehensive function .

shortcoming ： The throughput is relatively low , Message accumulation can seriously affect performance .erlang The language is not easy to customize .

Use scenarios ： Small scale scenes .

RocketMQ （ Ali products ）

advantage ： High throughput 、 High performance 、 High availability , Very comprehensive .

shortcoming ： The function of the open source version is not as good as that of the commercial version on the cloud . Official documents and surrounding ecology are not mature enough . The client only supports java
Use scenarios ： Almost the whole scene .

rabbitmq Principle of mirror queue ( Need to cluster )

How to ensure the order of messages ？

Global order and local order ：MQ Only need to ensure local order , There is no need to ensure global order .
The producer puts an ordered set of messages into the same queue , And consumers consume messages in the whole queue at one time .
RocketMQ There is a complete design in , But in RabbitMQ and Kafka among , There is no complete design , You need to design it yourself .
RabbitMQ： Make sure your goals exchange Only one queue . And a queue only corresponds to one consumer .
Kafka： Producers can customize partition Distribution rules , Assign messages to the same — individual partition.Topic Next only corresponds to one consumer .

rabbitmq The dead letter line 、 Delay queue principle

Delay queue can be called dead letter switch +ttl
Dead letter news ：

1. The message is denied and confirmed by the consumer , Use channel.basicNack or channel.basicReject, And at this point requeue Attributes are Set to false.
2. The message's lifetime in the queue exceeds the set TTL Time .
3. 3. The number of messages in the message queue has exceeded the maximum queue length . Then the message will become a dead letter message . If the dead letter queue information is configured , Then the message will be put into the dead letter queue , If not configured , Then The message will be discarded

Configure a dead letter switch for each service queue that needs to use dead letter , The dead letter switch of the same project can share one , Then for each business team The column is assigned a separate routeKey, The dead letter queue is just a queue bound to the dead letter switch , Dead letter exchange is not a special exchange machine , It's just a switch for receiving dead letters , So it can be for any type of 【Direct. Fanout. Topic]

TTL： Maximum lifetime of a message or all messages in the queue

If a message is set TL Properties or settings TTL Properties of the queue , So if this message is TTL The set time is not consumed , It will become ” Dead letter ”. If the queue is configured at the same time TTL And TTL, Then the smaller value will be used . Just need consumers to keep consuming the messages in the dead letter queue

rabbitmq Can I connect directly to the queue ？

The producer uses the same parameters as the consumer . Repeated declarations do not change the queue

RabbitMQ How to ensure the reliable transmission of messages
1、 Use transaction messages
2、 Using the message confirmation mechanism

Sender confirms ：

channel Set to confirm Pattern , Then each message will be assigned a unique id
Message delivered successfully , The channel will send ack To producers , Contains id, Callback ConfirmCallback Interface
If an error occurs, the message is lost , happen nack To producers . Callback ReturnCallback Interface
ack and nack Only one trigger , And only once , Asynchronous trigger . You can continue to send messages

The receiving party confirms that ：

When declaring a queue , Appoint noack=false, broker Will wait for the consumer to return manually ack、 Will delete the message , Otherwise, delete it immediately
broker Of ack There is no timeout mechanism , It will only judge whether the link is broken , If you disconnect 、 The message will be re sent

RabbitMQ How to ensure that messages are sent ？ Message reception ？( asynchronous )

Sender acknowledgement mechanism ：

The channel needs to be set to confirm Pattern , Then all messages published on the channel will be assigned a unique ID.

Once the message is delivered to queue（ Persistent messages need to be written to disk ）, The channel will send an acknowledgement to the producer （ Contains messages that are unique ID).

If RabbitMQ An internal error occurred that resulted in the loss of messages , Will send a nack（ Unconfirmed ） Message to producer .

All messages sent will be confirm（ namely ack) Or be nack once . But no response to the news was confirm Make any guarantee as fast as you can , and And the same message will not be both confirm Has been nack

The sender acknowledgement mode is asynchronous , The producer application is waiting for confirmation , You can send messages . When the confirmation message reaches the producer , Producer callback Method will be triggered .

Confirmcallback Interface ： Just make sure the incense arrives correctly Exchange in , If it arrives successfully, it will be called back Returncallback Interface ： Callback when the message fails to return

Receiver confirmation mechanism ：

When the consumer declares the queue , You can specify noAck Parameters , When noAck=false when ,RabbitMQ Will wait for the consumer to send back ack Signal from memory after （ Or disk , Remove messages from persistent messages . Xiangze , Messages are deleted immediately after they are consumed .

Consumers must confirm every message they receive （ Message reception and message acknowledgement are two different operations ）. Only the consumer confirms the message ,RabbitmQ To safely remove thoughts from the queue .

RabbitMQ Not for not ack Set the buy timeout for the message , It judges this elimination , Interest is the only basis that incense needs to be re delivered to consumers is the consumer who consumes the message Whether the connection has been disconnected . The reason for this design is RabbitwQ It can take a long time for consumers to eliminate a taboo . Ensure the final consistency of data ：
If the consumer returns ack The street opened a link before ,RabbitwQ Will be redistributed to the consumer of the next subscription .〈 There may be benefits of repeated consumption of messages , Need to be heavy ）
Sure key For the only id Storage reids in
( Ensure that the data is always , Current limiting ) tcp Long link

RabbitMQ Transaction message

RabbitMQ Transaction message

By setting the channel

1. channel.txSelect0; Notify the server to turn on transaction mode ; The server will return Tx.Select-ok
2. channel.basicPublish; Send a message , It can be multiple , It can be consumer message submission ack
3. 3. channel.txCommit0 Commit transaction ;
4. channel.txRollbacko Roll back the transaction ;

Consumer use transactions ：

1. autoAck=false, Manual submission ack, Subject to transaction commit or rollback ;
2. autoAck=true, Transaction is not supported , That is to say, it doesn't help that you roll back the transaction even after receiving the message , The queue has been cancelled The breath is removed except

If there is a problem in any of these links , Will throw loException abnormal , Users can intercept exceptions and roll back transactions , Or decide whether to re Reply to the message .

Transaction messages are reduced rabbitmq Performance of ( The transaction message is sent to the temporary queue )

RocketMQ How to ensure that no news is lost

producer ：

Send messages in synchronous blocking mode , Plus the failure retry mechanism , Probably broker Storage failure , It can be confirmed by query • Asynchronous sending requires rewriting the callback method , Check the sending result _ ack Mechanism , Maybe I'll save COrTTTictug, Storage ConsumerQueue Failure , Not visible to consumers at this time

broker： Synchronous brush set 、 Synchronous replication is adopted in cluster mode 、 Will wait for slave Confirmation will not be returned until the copy is completed

consumer ：

offset Manual submission , Message consumption guaranteed idempotent