Redis (17) -- redis cache related problem solving

13、 ... and ）Redis Cache related problems solved

13.1） Cache penetration

13.1.1） Problem description

key The corresponding data does not exist in the data source , Every time for this key The request for could not be fetched from the cache , Requests are pushed to the data source , That could crush the data source ; For example, use a non-existent user id Get user information , No matter cache or database , If hackers exploit this vulnerability, they may crush the database .

 

Cache penetration ：1) The application server is under more pressure 2）redis Hit rate reduced 3） Keep accessing the database to query information, so as to crash the database

Cache penetration reason ：1）redis No data to query 2） There are a lot of anomalies url visit 【 A malicious attack 】

13.1.2） Solution

There must be no data that cannot be cached or queried , Because the cache is written passively on Miss , And for the sake of fault tolerance , If no data can be found from the storage tier, it will not be written to the cache , This will cause the non-existent data to be queried by the storage layer every time it is requested , It loses the meaning of caching .

Solution ：

1. Cache null values ： If the data returned by a query is empty （ Whether the data doesn't exist or not ）, Still put this empty result （null） Cache , Setting the expiration time for empty results can be very short , Up to five minutes ;【 A temporary plan 】

2. Set up an accessible list （ White list ）： Use bitmaps Type defines a list of accessible , list id As bitmaps The offset , Every visit and bitmap Inside id Compare , If you visit id be not in bitmaps Inside , To intercept , Access not allowed ;【 The efficiency is not high 】

3. Use of Blum filter ： Hash all possible data to a large enough bitmaps in , A certain nonexistent data will be bitmaps Intercept , Thus, the query pressure on the underlying storage system is avoided ;

The bloon filter （Bloom Filter） yes 1970 Proposed by bron in . It's actually a very long binary vector ( Bitmap ) And a series of random mapping functions （ hash function ）; The bloom filter can be used to retrieve whether an element is in a collection . Its advantage is that the space efficiency and query time are far more than the general algorithm , The disadvantage is that it has certain error recognition rate and deletion difficulty

1. Real-time monitoring ： If I found Redis It's starting to drop , Need to check access objects and data , Cooperate with operation and maintenance personnel , You can set up a blacklist to restrict services

13.2） Cache breakdown

13.2.1） Problem description

key The corresponding data exists , But in redis Medium overdue , At this time, if a large number of concurrent requests come , These requests usually find that the cache is expired from the back end DB Load data and reset to cache , This is the time A large number of concurrent requests may instantly put the back end DB Overwhelmed .

  Cache breakdown ：1） Database access pressure increases instantaneously 2）redis There's not a lot of it key Be overdue 3）redis The normal operation

Cache breakdown reason ： redis One of the key Out of date , A large number of concurrent requests key, After the cache is found to be out of date, it is from DB Load cause DB Run away

13.2.2） Solution

key It may be accessed at some point in time with super high concurrency , It's a very “ hotspot ” The data of . This is the time , A question needs to be considered ： The cache is “ breakdown ” The problem of .

Solution ：

1. Preset hot data ： stay redis Before the summit visit , Put some hot data into redis Inside , Increase the hot data key Duration

2. Real time adjustments ： What data is hot on the spot , Real time adjustments key The expiration time of

3. Using locks ：

• When the cache fails （ Judge that the value is empty ）, Not immediately load db;

• First, use some operations of the cache tool with the return value of the successful operation （ such as Redis Of SETNX） Go to set One mutex key;

• When the operation returns success , Proceed again load db The operation of , And reset the cache , Finally delete mutex key;

• When the operation returns failure , Prove that there are threads in load db, The current thread sleeps for a period of time and then retries the whole get Caching method .

 13.3） Cache avalanche

13.3.1） Problem description

key The corresponding data exists , But in redis Medium overdue , At this time, if a large number of concurrent requests come , These requests usually find that the cache is expired from the back end DB Load data and reset to cache , At this time, a large number of concurrent requests may instantly put the back end DB Overwhelmed .

The difference between cache avalanche and cache breakdown is ： Cache avalanches are targeted at many key cache , Cache breakdown is some kind of key;

The normal access is shown in the figure below ：

  The cache avalanche moment is shown in the figure below ：

  Cache avalanche ：1） The pressure on the database is increasing 2） Server crash

Cache avalanche reason ：1） In very little time , A large number of key The centralized expiration of , Instant handle DB Overwhelmed

13.3.2） Solution

The avalanche effect of cache failure has a terrible impact on the underlying system ！

Solution ：

1. Building a multi-level cache architecture ：nginx cache + redis cache + Other caches （ehcache etc. ）;【 The architecture is complex 】

2. Use locks or queues ： Lock or queue to ensure that there will not be a large number of threads to read and write the database at one time , So as to avoid a large number of concurrent requests falling on the underlying storage system in case of failure ;【 Not for high concurrency 】

3. Set expiration flag to update cache ： Record whether the cache data is out of date （ Set the lead amount ）, If it expires, it will trigger another thread to update the actual situation in the background key The cache of ;

4. Spread the cache expiration time ： For example, we can add a random value to the original failure time , such as 1-5 Minutes at random , In this way, the repetition rate of each cache expiration time will be reduced , It's hard to trigger a collective failure .