Analysis of SEATA Distributed Transaction Framework

随着微服务框架的流行,At present, the design of application services is becoming more and more refined.Tasks that used to be done with a single app,At present, it may be split into different sub-services,Such as user center、数据中台、订单中心、Inventory platform.The ensuing question is how to ensure the consistency of data between the various sub-services,The need for distributed transactions is raised from it.

为什么需要事务

Before introducing distributed transactions,It is necessary to review the basic concepts of transactions again.

We know that the concept of transaction refers to“A series of operations either all success in the transaction,要么一个不做”,并且具有ACID四个基本特性：

• 原子性（Atomicity）：整个事务中的所有操作,要么全部完成,要么全部不完成,不可能停滞在中间某个环节.事务在执行过程中发生错误,会被回滚（Rollback）到事务开始前的状态,就像这个事务从来没有执行过一样.
• 一致性（Consistency）：在事务开始之前和事务结束以后,数据库的完整性约束没有被破坏.
• 隔离性（Isolation）：隔离状态执行事务,使它们好像是系统在给定时间内执行的唯一操作.If there are two transactions running to perform the same function at the same time,事务的隔离性将确保每一事务在系统中认为只有该事务在使用系统.这种属性有时称为串行化,为了防止事务操作间的混淆,必须串行化或序列化请求,使得在同一时间仅有一个请求用于同一数据.
• 持久性（Durability）：事务处理结束后,对数据的修改就是永久的,即便系统故障也不会丢失.

The above four characteristics are often used as computer test points,相信大家都不陌生.But let's think about it again,We usually use the transaction process,最重要的目的是什么？或者说ACID四者之中,Which one is our most concerned.

笔者认为,The reason why affairs have irreplaceable importance,Because of its consistency,也就是Consistency.

First look at atomicity and isolation,The core problem they solve is to ensure that data writing is consistent with program expectations in concurrent scenarios.But take a step back,We can downgrade all concurrent operations to serial execution by adjusting the program logic,To eliminate dependence on atomic and isolation.Although the efficiency may be greatly reduced,But at least from the application system point of view there are optional alternatives,For example, common distributed locks.

Persistence means that the data will continue to take effect after it is placed on the disk and will not be lost.But assume atomicity、On the premise that isolation and consistency are fully guaranteed,Persistence is also replaceable.例如Redis缓存,虽然RedisProvides a mechanism for data persistence,However, most of our scenarios will not rely onRedis做持久化存储,More often it is just to improve the efficiency of disaster recovery and recovery.Because the cached data is just a copy,On the premise that consistency is guaranteed,We can complete the replication of cached data from the data source.The data itself is complete,All we need to bear is the time cost of restoring the data.

Why consistency matters,It is because there is no means to ensure complete consistency from the application level,Must rely on transaction support.设想一个场景,Our code doesn't have anyBug,All operations are also proceeding as expected,But at some point database suddenly without electricity,Wait until power is restored,How do we get the program back to the running state at the moment of power outage？If there is no consistency guarantee,A large amount of inconsistent data will be generated after a power failure,例如：The user account has been debited but the order has not been generated、Commodity display has been lock but no corresponding orders.For this type of inconsistent data,Lossless failure recovery at the application level is almost impossible,Is because the application of the model：接收输入 -> Complete logical calculations -> 输出结果.If the data at the input is already inconsistent,The output results must also exist,Similar to the law of entropy increase in physics,Disorder is always amplified during processing.Consistency guarantees are beyond the scope of the application's design,Therefore, it can only be achieved through the transaction feature.

MySQLHow to achieve transactional consistency

接下来我们以MySQL的Innodb引擎为例,看看MySQLHow to achieve transaction consistency.

MySQL里面,All of the data to add and delete operations are not directly modify the data on the disk,Instead, the data is fetched to an in-memory buffer pool（Buffer Pool）,Then modify the content of the cache pool.This avoids the performance loss caused by frequent reading and writing to the disk,是MySQLCore mechanism to maintain high throughput.However, since memory operations cannot guarantee persistence,Once all the data in the operation when the power will be lost.对此MySQLTwo types of logs are used to deal with disaster recovery and recovery problems：RedoLog和UndoLog.

RedoLog是MySQLThe operation log that needs to be recorded before modifying the data page,Record which page files have been modified this time,and the modified result;UndoLog与RedoLog相对应,However, it records the content of the data record before modification.,And it is closer to the concept of table in data structure.从功能上看,RedoLogResponsible for ensuring the persistence of data writes,All forward modifications are recorded;UndoLogResponsible for ensuring data consistency,Reverse log of all modifications,确保能够通过UndoLogRestore data to the state before modification.

MySQLWhen modifying user data,始终遵循“日志先行”的规则,That is, before the actual operation of the data content,First ensure that all corresponding logs have been persisted,也就是RedoLog和UndoLog.而RedoLogPage storage structure and data storage structure is similar,也分为RedoLog Buffer和RedoLog File.Based on the above structure,A complete transaction commit process is as follows：

1. 开始事务;
2. Record modification dataUndoLog;
3. Modify the data cache pageData Buffer;
4. Record modification data的RedoLog Buffer;
5. Will in due timeRedoLog Buffer写入RedoLog File;
6. 完成事务提交;
7. Cache data when appropriateData Buffer写入磁盘文件Data File;

上述流程中,The key steps are：

• Perform a pre-modification recordUndoLog;
• Record after modifying the data cacheRedoLog;
• Make sure before the transaction commitsRedoLog Buffer写入RedoLog File.

These steps are guaranteed,All successful data modifications must correspond to a completeRedoLog和UndoLog;All unsuccessful or uncommitted transactions have a completeUndoLog.Therefore, even in the event of a power outage,It is also easy for us to judge how to complete data recovery based on the log.

• 场景一：事务已提交,But the data is not writtenData File
  从CheckPoint开始扫描RedoLog,It is found that the transaction has an operation log of the committed transaction,Play back this segmentRedoLog日志内容,Complete forward recovery;
• 场景二：事务未提交
  从CheckPoint开始扫描RedoLog,Found transaction has no commit step log or existsRollBack情况,则读取UndoLog,Complete reverse recovery.

An extra point of interest.前面提到了UndoLogThe data structure is closer to the table,是MySQLA more ingenious mechanism for disaster recovery and recovery.Because the structure is similar to the data table,在生成UndoLog的过程中,Modifications will also occurUndoLogcorresponding to the data pageRedoLog.因此,MySQLThe first stage of disaster recovery is always to restore firstRedoLog,Because when it just startedUndoLogPossibly also inconsistent;等所有RedoLogAfter processing is finished,此时UndoLogAlso be to restore the state of fault occurs,So the next stage will be based onUndoLogComplete failback.

Distributed transaction difficulties

Review the basic concept of the transaction,Next we enter the content of distributed transactions.

Distributed transaction is also a type of transaction,Therefore will carryACID的特性.But there is a problem,in a complex distributed environment,如何保证ACID同时满足.

First of all, several objective difficulties can be thought of：

一、Instability in network communication environment

Distributed transactions are scattered across different business systems due to the complete process,And each business system is deployed on different machines and equipment,Therefore, most of the time, it is necessary to communicate through the network cable..We know that the purpose of network protocols is to achieve reliable data delivery over unreliable transmission media,affected by the transmission medium,The higher the protocol level, the greater the cost to achieve reliability.上述MySQL通过RedoLog/UndoLog实现一致性,It is based on the physical circuit under the stand-alone architecture to complete the signal delivery,At this time, it is rare that the transaction is rolled back due to communication failure.,例如断电、硬盘故障、地震、火灾等;The distributed transaction is based on network communication,Regardless of the transmission rate of relegation,数据链路层、All exceptions at the transport layer,All may cause communication timeout、传输失败等.

二、Cluster size effect

假设一台机器24The probability of trouble-free operation in hours is99.9%,So four machines24The probability of trouble-free operation at the same time is99.6%,After expanding it to one hundred units,Its probability is only9%.A distributed transaction due to the nature of across applications,The complete link involves more machines involved,Thereby amplifying the risk of failure of the transaction process.

三、The impact of transaction isolation on performance in a clustered environment

为了确保一致性,The database will add an exclusive lock to the operation record when writing data,until the transaction commits or rolls back,The lock will always be held.But in a distributed transaction environment,This feature can become a fatal problem.Suppose the program calls the inventory interface for deduction,The next step is to call the order service to operate and encounter a network failure, causing it to hang,Then before this transaction times out and exits,All operations on inventory of the same item will block,If happened in seconds kill or core business processes,造成的损失将无法估量.

SEATA分布式框架

SEATA(Simple Extensible Autonomous Transaction Architecture)It is Alibaba's open source distributed transaction solution,Below is an excerpt fromSEATA官网的描述

Seata 是一款开源的分布式事务解决方案,致力于在微服务架构下提供高性能和简单易用的分布式事务服务.在 Seata 开源之前,Seata 对应的内部版本在阿里经济体内部一直扮演着分布式一致性中间件的角色,帮助经济体平稳的度过历年的双11,对各BU业务进行了有力的支撑.经过多年沉淀与积累,商业化产品先后在阿里云、金融云进行售卖.2019.1 为了打造更加完善的技术生态和普惠技术成果,Seata 正式宣布对外开源,未来 Seata 将以社区共建的形式帮助其技术更加可靠与完备.

SEATASupports four different distributed transaction modes,分别是AT、TCC、SAGA、XA模式.For the differences between the different modes, please refer to the previous分布式事务学习笔记这篇文章,这里主要介绍AT模式,也是SEATAUnique distributed transaction mechanism.

SEATA模型结构

在SEATAThere are three types of structural objects in the framework,are the transaction coordinators, respectivelyTC、事务管理器TM、资源管理器RM.

• TC (Transaction Coordinator) - 事务协调者
  维护全局和分支事务的状态,驱动全局事务提交或回滚.
• TM (Transaction Manager) - 事务管理器
  定义全局事务的范围：开始全局事务、提交或回滚全局事务.
• RM (Resource Manager) - 资源管理器
  管理分支事务处理的资源,与TC交谈以注册分支事务和报告分支事务的状态,并驱动分支事务提交或回滚.

实际部署时,TCis an independent distributed transaction control center,Can be deployed independently on a single machine,Can also be set up asTCClusters improve disaster tolerance;TMis the initiator of a distributed transaction,When a party initiates a distributed transaction,TMResponsible for completing the registration of global transactions、Commit and rollback;RMis a participant in a distributed transaction,It can be understood as each independent application in the microservice,When the participants completed a local transaction processing and ready to be submitted,将向TC注册分支事务,And wait for the final global submit/回滚.

AT模式介绍

相对于TCC、SAGA等模式,ATThe biggest feature of the model is that the business is non-invasive,Transaction processing is transparent to developers.下面是ATThe way the program initiates distributed transactions in mode：

@GlobalTransactional
public void purchase(String userId, String commodityCode, int orderCount) {
    
    ......
}

Just add it to the method [email protected]注解,That is, the start of the global transaction is completed,后续提交、Operations such as rollback are transparent to the program,Only need according to the normal business logic to complete the follow-up process development,There is almost no need to make special modifications to the business code.

However, the convenience of use does not mean that the underlying implementation is simple,Next, let's take a look at the implementation mechanism behind this annotation.

AT模式的核心

笔者认为ATThe pattern has three core elements：两阶段提交、global lock as wellUndoLog.

两阶段提交

ATThe pattern is essentially a two-phase commit（2PC）模型,The preparation phase and the commit phase.

第一阶段（准备阶段）：

1. TM发起全局事务,并在TC端注册,获取全局事务XID;
2. TM通过RPC调用子服务,并传递XID;
3. where the subservice is locatedRM执行本地事务,并生成相应UndoLog;
4. RM提交本地事务,向TCRegister branch transactions and report commit results;

第二阶段（提交场景）：

1. RM收到 TC 的分支提交请求,Put the request into an asynchronous queue,马上返回提交成功的结果给 TC;
2. RM异步删除UNDO LOG记录.

第二阶段（回滚场景）：

1. RM收到TCBranch rollback request,开启一个本地事务;
2. 通过XID和BranchId查找UndoLog;
3. 比对UndoLogPost image with current data,如果数据一致,执行下一步;Otherwise, it is processed according to the configuration policy;
4. 根据UndoLogFront mirror and businessSQLThe rollback statement is produced and executed;
5. 提交本地事务,And report the result of the local rollback transaction submissionTC;

比较特别的是,ATThe first stage of the mode has completed the commit of the branch transaction,That is, each service data persistence has been completed,When committing globally at this time,No further processing required for the second stage,Just clean up the first stageUndoLog即可.And when the global rollback,Each sub-service needs to go throughUndoLogContent initiates local transaction again,Restore data to the state it was in before the global transaction.

因此AT模式可以理解为：本地事务 + 全局调度.Use local transactions of each sub-service to complete data writing,At the same time, the branch transaction is registered with the global before the local transaction is committed,Received global commit/After rolling back the command, judge whether it is necessary to restore this modification.

全局锁

由于ATThe first phase of the pattern has completed the local transaction commit,Therefore, all database locks are released as soon as the local transaction is executed..But according to businessACID特性,At this time, additional mechanisms are required to ensure that 本地事务提交后 -> 全局事务提交 期间,Data modified by all transactions cannot be modified by other transactions.对此,这里ATThe mode uses the mechanism of the global lock.

The global lock here refers to the distributed lock,It ensures write isolation for global transactions,and read isolation in specific scenarios.Its mechanism is as follows：

写隔离：

• 一阶段本地事务提交前,需要确保先拿到全局锁.
• 拿不到全局锁,不能提交本地事务.
• 拿全局锁的尝试被限制在一定范围内,超出范围将放弃,并回滚本地事务,释放本地锁.

读隔离：

• 仅针对Select For Update场景,Apply for a global lock through an aspect proxy;
• The default isolation level is read uncommitted（Read Uncommitted）

下面通过SEATAExamples on the official website to understand the function of global locks.

1. 全局事务tx1先执行,After it starts the local transaction,First obtain the local lock to modify the record,and complete the data update;
2. tx1提交本地事务前,First acquire the global lock on the modified record,After getting it, the local transaction commit is completed,释放本地锁;
3. 全局事务tx2后执行,It also starts local transactions,First try to acquire a local lock on the same record;此时由于tx1Local transaction commit completed,因此tx2The local transaction acquired the lock successfully;
4. tx2Complete data modification,Before preparing to commit a local transaction,will also try to acquire the global lock on the modified record,However, since the global lock istx1持有,因此tx2Need to wait to retry global lock,直到tx1提交.

The feature of global locks ensures write isolation between global transactions,So you don't have to worry about dirty writes.The above is the scene of normal submission,仔细观察,在tx2Submit a local transaction when there is a phenomenon：此时tx2Hold local lock and wait for global lock,tx1Hold global lock and wait for commit/回滚.那假设tx1Then there was a rollback,根据前面2PC的步骤描述,tx1Need to start a local transaction data rolled back,But this time the local lock istx2所占有,于是死锁就发生了.对此SEATA给出的方案是tx1The branch rollback will keep retrying because the global lock cannot be obtained,直到tx2Abandon the request and roll back the local transaction after the request for the global lock times out,从而解决死锁.

The implementation mechanism of the global lock

Global locks are essentially distributed locks,Therefore, we can check the content of the lock intuitively..在SEATAThe transaction coordinatorTCThe core part consists of three tables：

• global_table：每当有一个全局事务发起后,就会在该表中记录全局事务的ID
• branch_table：记录每一个分支事务的ID,分支事务操作的哪个数据库等信息
• lock_table：记录全局锁,Currently holds a global transactionID、分支事务ID、数据库主键

其中lock_tableIt is the global lock data table,我们可以通过将SEATA配置文件的store.mode配置为db,That is, the relevant content of the current global transaction can be queried in real time through the database.The following is the global transaction during the execution of the distributed transaction、Screenshot of branch transaction and global lock.

全局事务：

分支事务：

全局锁：

It can be seen that the core of the global lock is to record the global transaction that currently holds the lockID（XID）,以及对应RMdatabase primary key.

UndoLog

最后,支持ATThe core mechanism that can complete data rollback without intrusion in the mode,就是SEATA的UndoLog.

这里思考一个问题,为什么SEATA不需要RedoLog？因为SEATA不像MySQL一样存在Data Buffer -> Data Filepersistence structure,As long as the local transaction commits successfully,That is, it can be considered that the branch transaction data has been persisted,No need to consider how to ensure data forward recovery in disaster recovery scenarios,These are guaranteed by the local transaction of the database.因此SEATAThe core of it is how to restore data before the branch transaction occurs,而这里SEATASo is the mechanism usedUndoLog.

与数据库的UndoLog不同,SEATADefine your own on top of the databaseUndoLog数据格式,The advantage of this is to avoid the need for different database platformsUndoLogParsing problems caused by different formats,At the same time, it also provides convenience for people to intervene.

下面是SEATA中UndoLog的表结构定义：

可以看到其结构非常简单,The core content isrollback_info的字段,The storage inside is structuredJSON字符串,The pre-image and post-image of the data modification are recorded,Corresponding to the content before modification and the content after modification,如下.

{
    
    "@class":"io.seata.rm.datasource.undo.BranchUndoLog",
    "xid":"192.168.0.106:8091:8313831052107195637",
    "branchId":8313831052107195639,
    "sqlUndoLogs":[
        "java.util.ArrayList",
        [
            {
    
                "@class":"io.seata.rm.datasource.undo.SQLUndoLog",
                "sqlType":"UPDATE",
                "tableName":"t_storage",
                "beforeImage":{
    
                    "@class":"io.seata.rm.datasource.sql.struct.TableRecords",
                    "tableName":"t_storage",
                    "rows":[
                        "java.util.ArrayList",
                        [
                            {
    
                                "@class":"io.seata.rm.datasource.sql.struct.Row",
                                "fields":[
                                    "java.util.ArrayList",
                                    [
                                        {
    
                                            "@class":"io.seata.rm.datasource.sql.struct.Field",
                                            "name":"id",
                                            "keyType":"PRIMARY_KEY",
                                            "type":4,
                                            "value":1
                                        },
                                        {
    
                                            "@class":"io.seata.rm.datasource.sql.struct.Field",
                                            "name":"count",
                                            "keyType":"NULL",
                                            "type":4,
                                            "value":976
                                        }
                                    ]
                                ]
                            }
                        ]
                    ]
                },
                "afterImage":{
    
                    "@class":"io.seata.rm.datasource.sql.struct.TableRecords",
                    "tableName":"t_storage",
                    "rows":[
                        "java.util.ArrayList",
                        [
                            {
    
                                "@class":"io.seata.rm.datasource.sql.struct.Row",
                                "fields":[
                                    "java.util.ArrayList",
                                    [
                                        {
    
                                            "@class":"io.seata.rm.datasource.sql.struct.Field",
                                            "name":"id",
                                            "keyType":"PRIMARY_KEY",
                                            "type":4,
                                            "value":1
                                        },
                                        {
    
                                            "@class":"io.seata.rm.datasource.sql.struct.Field",
                                            "name":"count",
                                            "keyType":"NULL",
                                            "type":4,
                                            "value":973
                                        }
                                    ]
                                ]
                            }
                        ]
                    ]
                }
            }
        ]
    ]
}

在SEATA里,UndoLogIt is by analyzing the businessSQL得来的.当RMwhile in a global transaction,The following steps will be followed to execute the local transaction：

1. 拦截DML语句,Parse its filter conditions;
2. Extract the primary key of the data that needs to be modified in this operation;
3. Save the data content before modification,Form the front mirror;
4. 执行DML语句,Complete data modification;
5. Save the modified data content,post-mirror;
6. Summarize the content of pre-image and post-image,写入UndoLog;
7. 提交本地事务;

利用UndoLog,Branch transactions can be in a global transaction rollback changes on all matters of branch of modified reduction to a state of affairs at the beginning of the operation.However, readers may have noticed,When rolling back, only the previous image is needed to complete data restoration,Why do you need to record the post-image after data modification?？

这是SEATAA data verification mechanism left,As mentioned earlier, the essence of global locks is distributed locks,那就会存在一个问题,Data isolation takes effect only for modification operations that understand and abide by distributed lock rules,If the party operating the data is not within this mechanism,data isolation cannot be guaranteed.例如DBAManual correction,这个时候SEATADoes not perceive that data has been modified outside the framework,Only inconsistencies are found by comparing the current data with the post-image content during the rollback process,At this time, manual or pre-configured policies are required for special processing.与此类似,If all business systems corresponding to the database cannot be incorporated into the distributed transaction framework,ATThe schema will also not be able to guarantee its consistency,This is also somewhat limitedAT模式的使用场景.

SEATA的高可用设计

The core of distributed transaction flow isTC,为了防止TCbecome an architectural bottleneck,SEATAThrough the concept of grouping will join affairsTCService traffic is offloaded to different nodes,So as to achieve high availability support.具体可见SEATA官网的相关介绍.

总结

SEATA的ATThe mode achieves a very good feature of no business intrusion,Simple to use and less costly to retrofit,不过即便如此,Distributed transactions have not yet been widely used.究其原因,The author thinks that mainly distributed environment next roll cost is too high,结果不可控.We use the transaction is the core of in order to ensure consistency,Local transactions are out of the scope of application control,Therefore, it cannot be replaced by architectural design.Distributed transactions are special,The inconsistency is not in the program input and output,And data asymmetry between different systems,But this asymmetry can be solved within the framework of the program.

根据CAP与BASE理论,Distributed systems pursue eventual consistency,Ensure that the system can eventually reach a consistent state after a certain period of synchronization;And before that the system can be loose,A certain degree of inconsistent state is allowed in the middle.Distributed transactions are satisfiedCP特性,但在可用性Amade concessions,So not all scenarios are applicable,需要有所取舍.But as said before,If all related business systems are not included in the distributed transaction framework,Consistency guarantees will be greatly reduced.So technology is neither good nor bad,Only see if it can meet the business scenario.

作者：Ruan Weicong