Data model and operation interface

Start building SimpleKV when , First of all, we should consider what kind of data can be stored in it , What operations can be done on the data , That is, data model and operation interface .
The data model and the operation interface can decide under what circumstances it is suitable to use the key value database constructed by us , Under what circumstances is not suitable for , Instead, you need to use other types of databases .

Data model

The basic data model of key value database is ： key-value Model .
Usually key yes String type .

Some key value databases only support vlaue by String type , for example memcached database .
It can also be designed as value All basic data types are supported +String type .
and redis Multiple complex data types are supported ： Such as String、 Hashtable 、 list 、 Collection etc. .

Redis It can be widely used in actual business scenarios , It benefits from supporting diversified types of value.

Operation interface

newly added \ to update ：PUT( perhaps SET)
obtain ：GET
Delete ：DELETE
Range ：SCAN
Does it already exist ：EXISTS

Whether key value pairs are stored in memory or external memory ？

Save in memory , Fast access (ns Level ), There is a risk of power failure and loss
Save in external storage , Power off will not be lost , But the access speed is too slow (ms Level ).

Select according to the application scenario

for example , The data in the cache scenario is required to be accessed quickly, but it is allowed to be lost , At this time, the key value data is saved in memory .
Memcached and Redis Are all memory key value databases .

What access mode is used ？- Access module

One way is to use it for external applications through function library calls , such as , Link to our own program in the form of dynamic link library .
You can also use the network framework to Socket Provide key value pair operation in the form of communication .

Network framework access mode

The mode of providing access services through the network framework , The application scenario of key value database is expanded , But it also gives the performance of key value database 、 The design of the operation model brings some potential problems ： How to design a suitable network I\O Model ？

The so-called network I\O, In the network framework access mode , How to handle user access ： How to handle network connection 、 What protocol is used to resolve network requests 、 How to handle data access ？ Whether to use single thread or multi thread or multi process ？ Different networks I\O The model has different effects on the performance and scalability of key value database .

If you use a thread, you have to deal with both network connections 、 Request parsing , And complete data access , So once a certain operation is blocked 、 The whole user access will be blocked , This reduces the system response speed .
If we use different threads to handle different operations , So when a thread is blocked , Other threads can still run normally . But when multiple threads access the same resource , There will be competition again , It also affects efficiency .

Redis Using a single threaded network I\O Model , And did it “ Single thread 、 High performance ”.

How to locate key value pairs - Index module

When the user request is parsed , The key value database needs to find out whether the key value pair requested by the user exists , This depends on the index module of the database .
The function of the index is , Let the key value database according to key find value Storage location , And then perform the operation .

There are many types of indexes , The common ones are hash tables 、B+ Trees 、 Dictionary tree, etc .
Different index types consume space 、 performance 、 Concurrency control has different characteristics .
Some key value databases use jump tables to store key-value Yes , and redis and memcached Are all hash tables .

Send the user requested key-value Write or delete - Storage module

When the user calls PUT When the command operates on data , Need to be for key-value Allocate memory space for data
When the user calls DELETE When the command operates on data , Need to put key-value Free up the occupied memory space

Memory allocator

SimpleKV Simple glibc As a memory allocator .
But for Redis Come on , Because of its value Support various complex data types , Therefore, the size of key value pairs varies ,glibc When dealing with random size memory block allocation , Not so good . Once the saved key value is too large for the data , There will be a serious memory fragmentation problem .

Redis A variety of optional memory allocators are provided , Users can choose according to the application scenario .

How to provide services quickly after restart - Persistence

There are two persistence schemes ：
Every time PUT perhaps DELETE when , Drop disk while writing memory （ Call the file interface to write the file ）. Will result in very low efficiency
Periodically drop the data to the disk according to a certain frequency , Every once in a while, all the PUT perhaps DELETE Write the data to the file , In this case, it is possible to lose