How to implement distributed locks with redis?

brief introduction

   I believe that the biggest motivation for many people to learn distributed lock is not the need of their own system , But the interviewer needs ... Of course , This also shows that the distribution lock is very important , Often used as test questions , Before learning , We need to clarify a few questions first .

One 、 Is the lock important ？

   Of course important. , When accessing critical resources , Locks are used , Otherwise, there will be thread safety problems .

Two 、 Then why don't we Java The lock that comes with it ？ such as synchronized and Lock I have to realize it myself ？

   Here's a clear question , these Java The self-contained locks are all in the same JVM That's what works , In distributed services , There will be multiple JVM Concurrent access to services under virtual machines , These locks don't work . Distributed environment , Locks require third-party services .

3、 ... and 、 What are the commonly used distributed lock implementation schemes ？

• be based on MySQL Distributed locks for
• be based on Redis Distributed locks for
• be based on ZooKeeper Distributed locks for

   In fact, it's plain , As long as the database that can store data can realize distributed locking , Because we just need to tell other threads , If the current resources are occupied , Actually synchronized and Lock It also stores a tag , Tell other threads , Is the current resource occupied , There is no mystery .

Three ways of implementation

   First we need to understand , What requirements should distributed locks meet ：

1. Mutual exclusivity .( In a distributed cluster , The same method can only be obtained by one thread on one machine at the same time ）.
2. Reentrancy .( Recursive calls should not be blocked 、 Avoid deadlock ）.
3. Lock timeout .( Avoid deadlock 、 Dead loop and other unexpected situations ).
4. Locking and unlocking must be the same client .（ Unless the lock expires and automatically retracts , Otherwise, locking and unlocking need to be the same client ）.

Next, we briefly introduce the implementation principles of the next three locks , And their advantages and disadvantages .

be based on MySQL Distributed locks for

   be based on MySQL There are two main ways to implement locks , One is Pessimistic locking , One is Optimism lock .

1. Pessimistic locking ： Mainly used select … where … for update Lock and operate the queried line , It should be noted that “where name = lock ”,name Fields must be indexed , Otherwise, it will lock the watch .
2. Optimism lock ： Is based on CAS Thought , I don't think lock contention often happens , Only update version It's not until you fail that you realize , Lock contention is not long , It's a good solution , But too much competition , It's a waste CPU resources .

advantage ：

• Implementation is relatively simple , All by MySql To solve the problem of competition .
• The architecture is relatively simple , No longer need redundant third-party components , Make the whole system simpler .

shortcoming ：

• Relatively poor performance , And there's the risk of locking the table .
• After a non blocking operation fails , Need to poll , Occupy cpu resources , and MySQL Database resources .
• Long time non submission or long time polling , Will take up too much MySQL Connect resources .

be based on Redis Distributed locks for

  Redis Some operations and features of , This article may be a reference ：Redis Learning notes . Here is a direct introduction to the three commands that need to be used , Namely ：SETNX、expire、delete.

1. SETNX key val
   SETNX by SET if Not eXists abbreviation , The role is ： if key non-existent , Then save the key value pair , return 1; if key There is , Then do nothing , return 0.
2. expire key <timeout>
   by key Set a timeout , Unit is second, After this time, the lock will be automatically deleted key, Avoid deadlock caused by downtime .
3. delete key
   Delete the specified key.

Realize the idea of ：

1. When getting the lock , Use setnx Lock ,key Is the lock name ,value Value is a randomly generated UUID.
2. After obtaining the lock successfully , Use expire Command to add a timeout to the lock , If it exceeds this time, give up acquiring lock .
3. When you release the lock , adopt UUID Decide whether to lock , If it's time to lock , execute delete Lock release .

advantage ：

• Depend on Redis High concurrency features , So the performance is excellent .
• Expiration time is not easy to control , We need to consider the problem of continued locking .

shortcoming ：

• The implementation is relatively complex , There are too many factors to consider .
• Non blocking , After the operation fails , Need to poll , Occupy cpu resources .
• The master node is down , In case of unsuccessful synchronization , It may cause multiple nodes to acquire locks .

be based on ZooKeeper Distributed locks for

  ZooKeeper The data storage data model is a tree （Znode Tree), By slash （/） The path of segmentation , It's just one. Znode（ Such as /locks/my_lock）. Every Znode They will save their own data content on the Internet , At the same time, a series of attribute information will be saved .Znode It can be divided into four types ： Persistent node 、 Persistent order nodes 、 Temporary node 、 Temporary order node .
  ZooKeeper Distributed locks are based on Temporary order node To achieve , Lock can be understood as ZooKeeper A node on , When a lock needs to be acquired , Create a temporary sequence node under this lock node . When there are multiple clients to acquire locks at the same time , Create multiple temporary order nodes in order , But only the node whose permutation number is the first can acquire the lock successfully , Other nodes listen to the changes of the previous node in order , When the listener releases the lock , The listener can get the lock immediately .
   Also used Temporary order node Another intention of is , If a client creates Temporary order node after , It doesn't matter if you accidentally go down ,ZooKeeper When a client is perceived to be down, it will automatically delete the corresponding temporary order node , Equivalent to automatic release lock .

advantage ：

• Effectively solve the problem of single point of downtime , High availability .
• It can be used as a reentrant lock .
• It can be used as a blocking lock .

shortcoming ：

• You need to create and delete nodes frequently , Not as good as Redis.
• The client and Zookeeper Long lost contact , Lock release problem .

summary

In terms of ease of implementation ：MySQL database > Zookeeper > Redis cache .
In terms of performance ：Redis cache > Zookeeper > MySQL database .
Comparison in reliability ：Zookeeper > MySQL database > Redis cache .

Redis The specific implementation code of

   use Redis When doing code implementation , We need to consider the following questions ：

• Deadlock caused by downtime ： Set expiration time .
• Business time is longer than expiration time ： Open a daemon , Do lock renewal .
• The lock was released by someone else ： Lock write unique ID , Check the identification before releasing the lock , Re release .

So what if it's too troublesome to implement , Directly use the library encapsulated by others ：Redisson. Not only is it more convenient , And more stable , The code is as follows ：

// 1. structure redisson It is necessary to implement distributed lock Config
Config config = new Config();
config.useSingleServer().setAddress("redis://127.0.0.1:6379").setPassword("password").setDatabase(0);

// 2. structure RedissonClient
RedissonClient redissonClient = Redisson.create(config);

// 3. Get lock object instance 
RLock rLock = redissonClient.getLock(lockKey);
try {
   
    // 4. Attempt to acquire lock 
    boolean res = rLock.tryLock((long)waitTimeout, (long)leaseTime, TimeUnit.SECONDS);
    if (res) {
        //  Lock success , Deal with business 
    }
} catch (Exception e) {
    //  Continue to wait for , Or do something else 
    throw new RuntimeException("aquire lock fail");
}finally{
    //  in any case ,  Finally, we need to unlock 
    rLock.unlock();
}

Redisson The implementation method is as follows ：
tryLock The implementation method is as follows ：

	public boolean tryLock(long waitTime, long leaseTime, TimeUnit unit) throws InterruptedException {
        long time = unit.toMillis(waitTime);
        long current = System.currentTimeMillis();
        long threadId = Thread.currentThread().getId();
        
        //  Attempt to acquire lock 
        Long ttl = tryAcquire(leaseTime, unit, threadId);
        // lock acquired
        if (ttl == null) {
            return true;
        }

        //  If the time taken to apply for a lock is greater than or equal to the maximum waiting time , The lock application failed .
        time -= System.currentTimeMillis() - current;
        if (time <= 0) {
            acquireFailed(threadId);
            return false;
        }

        current = System.currentTimeMillis();

        /**
         *  Subscribe to lock release events , And pass  await  Method blocks waiting for the lock to be released , It effectively solves the problem of waste of resources in invalid lock application ：
         *  When  this.await  return  false, Indicates that the waiting time has exceeded the maximum waiting time for obtaining the lock , Unsubscribe and return lock acquisition failure .
         *  When  this.await  return  true, Enter the loop to try to get the lock .
         */
        RFuture<RedissonLockEntry> subscribeFuture = subscribe(threadId);
        // await  The internal method is to use  CountDownLatch  To block , obtain  subscribe  The result of asynchronous execution （ Applied  Netty  Of  Future）
        if (!subscribeFuture.await(time, TimeUnit.MILLISECONDS)) {
            if (!subscribeFuture.cancel(false)) {
                subscribeFuture.onComplete((res, e) -> {
                    if (e == null) {
                        unsubscribe(subscribeFuture, threadId);
                    }
                });
            }
            acquireFailed(threadId);
            return false;
        }

        try {
            //  Calculate the total time taken to acquire the lock , If greater than or equal to the maximum waiting time , Failed to acquire lock .
            time -= System.currentTimeMillis() - current;
            if (time <= 0) {
                acquireFailed(threadId);
                return false;
              }

            /**
             *  After receiving the lock release signal , Within the maximum waiting time , Loop through one attempt after another to get the lock 
             *  Lock acquired successfully , Then immediately return to  true,
             *  If the lock has not been acquired within the maximum waiting time , It is considered that the acquisition of lock failed , return  false  End of cycle 
             */
            while (true) {
                long currentTime = System.currentTimeMillis();

                //  Try to acquire the lock again 
                ttl = tryAcquire(leaseTime, unit, threadId);
                // lock acquired
                if (ttl == null) {
                    return true;
                }
                //  If the maximum waiting time is exceeded, return  false  End of cycle , Lock acquisition failed 
                time -= System.currentTimeMillis() - currentTime;
                if (time <= 0) {
                    acquireFailed(threadId);
                    return false;
                }

                /**
                 *  Block wait lock （ Through semaphores ( Shared lock ) Blocking , Wait for unlock message ）：
                 */
                currentTime = System.currentTimeMillis();
                if (ttl >= 0 && ttl < time) {
                    // If there is time left (ttl) Less than wait time , It's just  ttl  Within time , from Entry Get a license for the semaphore ( Unless interrupted or no license is available ).
                    getEntry(threadId).getLatch().tryAcquire(ttl, TimeUnit.MILLISECONDS);
                } else {
                    // It's in wait time  Waiting in the time range can pass through the semaphore 
                    getEntry(threadId).getLatch().tryAcquire(time, TimeUnit.MILLISECONDS);
                }

                //  Update remaining wait time ( Maximum waiting time - Blocking time that has been consumed )
                time -= System.currentTimeMillis() - currentTime;
                if (time <= 0) {
                    acquireFailed(threadId);
                    return false;
                }
            }
        } finally {
            //  Whether or not a lock is obtained , You have to unsubscribe from the unlock message 
            unsubscribe(subscribeFuture, threadId);
        }
        return get(tryLockAsync(waitTime, leaseTime, unit));
    }

Redisson Watchdog lock renewal mechanism

	private <T> RFuture<Long> tryAcquireAsync(long leaseTime, TimeUnit unit, long threadId) {
		
		//  If it has an expiration date , The lock is obtained in the normal way 
        if (leaseTime != -1) {
            return tryLockInnerAsync(leaseTime, unit, threadId, RedisCommands.EVAL_LONG);
        }

        //  First according to 30 Seconds to execute the lock acquisition method 
        RFuture<Long> ttlRemainingFuture = tryLockInnerAsync(commandExecutor.getConnectionManager().getCfg().getLockWatchdogTimeout(), TimeUnit.MILLISECONDS, threadId, RedisCommands.EVAL_LONG);
        
        //  If you still have this lock , Then the timer task is opened to refresh the expiration time of the lock 
        ttlRemainingFuture.onComplete((ttlRemaining, e) -> {
            if (e != null) {
                return;
            }

            // lock acquired
            if (ttlRemaining == null) {
                scheduleExpirationRenewal(threadId);
            }
        });
        return ttlRemainingFuture;
    }

Renewal principle
Just use lua Script , Reset the lock time to 30s

/*
 Watch Dog  The mechanism is actually a background timed task thread , After obtaining the lock successfully , The thread holding the lock will be put into a  RedissonLock.EXPIRATION_RENEWAL_MAP Inside ,
  Then every  10  second  （internalLockLeaseTime / 3）  Check the , If the client   And hold the lock  key
 （ Determine whether the client still holds  key, It's actually traversal  EXPIRATION_RENEWAL_MAP  Inside the thread  id  And then according to the thread  id  Go to  Redis  Intermediate investigation , If it exists, it will prolong  key  Time for ）,
  Then it will keep extending the lock  key  Survival time . If the service goes down ,Watch Dog  There is no thread mechanism ,
  There will be no extension at this point  key  The expiration time of , here we are  30s  Then it will automatically expire , Other threads can acquire the lock .
*/
private void scheduleExpirationRenewal(long threadId) {
    ExpirationEntry entry = new ExpirationEntry();
    ExpirationEntry oldEntry = EXPIRATION_RENEWAL_MAP.putIfAbsent(getEntryName(), entry);
    if (oldEntry != null) {
        oldEntry.addThreadId(threadId);
    } else {
        entry.addThreadId(threadId);
        renewExpiration();
    }
}

protected RFuture<Boolean> renewExpirationAsync(long threadId) {
    return commandExecutor.evalWriteAsync(getName(), LongCodec.INSTANCE, RedisCommands.EVAL_BOOLEAN,
            "if (redis.call('hexists', KEYS[1], ARGV[2]) == 1) then " +
                "redis.call('pexpire', KEYS[1], ARGV[1]); " +
                "return 1; " +
            "end; " +
            "return 0;",
        Collections.<Object>singletonList(getName()),
        internalLockLeaseTime, getLockName(threadId));
}