当前位置:网站首页>maxwell concept
maxwell concept
2022-08-11 07:26:00 【Eating too much sugar will not gain weight】
maxwell concept
maxwell definition
maxwell is a real-time capture MYSQL software written in java. It reads the MYSQL binary log Binlog in real time and generates JSON format, which is used as a producer to send to Kafka, Redis and other applications.
The Principle of Maxwell
Disguise yourself as a MYSQL salve, and then copy data from the master as a salve.
Mysql master-slave replication process
The master main library change record is written to the binary log file.
salve sends the dump protocol from the library I/O thread to the mysql master, and copies the binary log events of the master master library to the relay log of the slave library.
salve has a sql thread that will read the relay log,The sql events in this log are executed sequentially to be consistent with the main library.
binary log classification
binary log has three formats: statement, row, mixed
statement
At the statement level, binlog records the statement of the write operation each time, which saves space, but may produce different results, such as update test set create_date=now(). If you use binlog to reply to the log, the execution time will also be different., the resulting data is different.
row
Row level, binlog records the changes of each row of records after each operation, advantages: maintain data consistency, because no matter what operation is performed, it only records the effect after execution.Disadvantage: takes up a lot of space
mixed
Mixed level, the upgraded version of statement, the default is statement, but when the function contains UUID(); when the table containing the AUTO_INCREMENT field is updated; when the INSERT DELAYED statement is executed; when using UDF; it will be processed in the way of ROW.Advantages: It saves space and also takes into account a certain consistency.Disadvantages: There are also individual cases that can cause inconsistencies.
binlog format chosen by maxwell
Maxwell needs to monitor and analyze, and it is more suitable to choose the row format, which has strong consistency.
maxwell and canal comparison
Language: All written in java
Data format: canal format is free, maxwell can only send json messages
Data collection mode: canal only supports incremental collection, maxwell supports full and incremental
Data landing: Canal needs to be customized, maxwell supports kafka and other platforms
ha: all support
边栏推荐
- PIXHAWK飞控使用RTK
- JD.com product details API call example explanation
- 皮质-皮质网络的多尺度交流
- 【推荐系统】:协同过滤和基于内容过滤概述
- mmdetection的安装和训练、测试didi数据集的步骤(含结果)
- 下一代 无线局域网--强健性
- OA project meeting notice (query & whether attending & feedback for details)
- Spatial Pyramid Pooling -Spatial Pyramid Pooling (including source code)
- 基于FPGA的FIR滤波器的实现(4)— 串行结构FIR滤波器的FPGA代码实现
- 每日sql - 判断+聚合
猜你喜欢
导航定位中的坐标系
Pinduoduo api interface application example
拼多多api接口应用示例
Spatial Pyramid Pooling -Spatial Pyramid Pooling (including source code)
Douyin get douyin share password url API return value description
ROS 话题通信理论模型
下一代 无线局域网--强健性
My meeting of the OA project (meeting seating & review)
什么是Inductive learning和Transductive learning
daily sql - query for managers and elections with at least 5 subordinates
随机推荐
京东商品详情API调用实例讲解
unable to extend table xxx by 1024 in tablespace xxxx
maxwell 概念
Do not add the is prefix to the variables of the boolean type in the POJO class of the Alibaba specification
MySQL 版本升级心得
损失函数——负对数似然
PIXHAWK飞控使用RTK
Redis源码:Redis源码怎么查看、Redis源码查看顺序、Redis外部数据结构到Redis内部数据结构查看源码顺序
淘宝API常用接口与获取方式
每日sql -查询至少有5名下属的经理和选举
你是如何做好Unity项目性能优化的
My approval of OA project (inquiry & meeting signature)
Top20 bracket matching
Trill keyword search goods - API
Edge 提供了标签分组功能
【推荐系统】:协同过滤和基于内容过滤概述
Unity程序员如何提升自己的能力
Concurrent programming in eight-part essay
Taobao API common interface and acquisition method
Conference OA Project My Conference