Design and practice of the smallest short website system in the whole network

Introduce

Today, we introduce the design and implementation of a short website system . The so-called short link is that no matter how long your link is , Eventually it will generate a Fixed length short links . Although the obligation is simple , But it involves a lot of details . Ensure short links only and Access speed Become a core issue , Then start performing .
Short link application scenario ：

• Marketing : Text messaging 、 Mobile link forwarding （ Bili, Bili ）
• Article content layout
• Enterprises send interview questions
• Voting and questionnaires

One 、 Related concepts and technologies

1.1 Redis cache

In this system , We will use redis data type Hash, Used to store long links we want to access , We all know ,Hash Query for Time complexity by O（1）, Therefore, it is used to preserve code And a long link KV Correspondence can improve the efficiency of short chain to long link Forwarding speed （302 agreement ）, The storage model is as follows ：

1.2 Generate 6 Bit random code algorithm

The core of short URL implementation is to ensure the generation of 6 position code Must be unique , Using this algorithm can not completely guarantee the uniqueness , So I designed it in Obligation layer introduce redis Advanced data types The bloon filter Made an optimization , To ensure that the generated 6 Bit code is unique .

public class ShortUrlCodeUtil {
    
    private static final String randomStr = "hnmnd0d";

    public static String genShortCode(String url, String randomStr) {
    
        return shortUrl(url, randomStr)[0];
    }

    public static String genShortCode(String url) {
    
        return shortUrl(url, randomStr)[0];
    }

    private static String[] shortUrl(String url, String randomStr) {
    
        String[] chars = new String[]{
    "a", "b", "c", "d", "e", "f", "g", "h",
                "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s", "t",
                "u", "v", "w", "x", "y", "z", "0", "1", "2", "3", "4", "5",
                "6", "7", "8", "9", "A", "B", "C", "D", "E", "F", "G", "H",
                "I", "J", "K", "L", "M", "N", "O", "P", "Q", "R", "S", "T",
                "U", "V", "W", "X", "Y", "Z"
        };
        String encrypt = (MD5.encrypt(randomStr + url));
        String hex = encrypt;
        String[] resUrl = new String[4];
        for (int i = 0; i < 4; i++) {
    
            String t = hex.substring(i * 8, i * 8 + 8);
            long lHex = 0x3FFFFFFF & Long.parseLong(t, 16);
            String charTemp = "";
            for (int j = 0; j < 6; j++) {
    
                long index = 0x0000003D & lHex;
                charTemp += chars[(int) index];
                lHex = lHex >> 5;
            }
            resUrl[i] = charTemp;
        }
        return resUrl;
    }
}

1.3 The bloon filter

Think about the problem ：

• Why use a bloom filter , Don't use set aggregate ？
• What are the obligations of Bloom filter ？
• The bottom principle of Bloom filter ？

1. This system uses The bloon filter The purpose of is to generate 6 position code duplicate removal , Prevent duplicate codes , Although the probability of repetition is very small , But we should still prevent , So I made a Optimize , As for the same weight removal , Why not set, For reasons of length , Don't expand .
2. secondly , Bloom filter has another function in this system , That's the solution Cache hit penetration problem , When a user sends a short link request , First, the system will analysis Out 6 position code, Then go first The bloon filter lookup , If There is Go again cache obtain code Corresponding Long link , Cache does not exist, go again database lookup ; If non-existent Words , Throw an exception directly , No more database and cache queries .

@Service
public class ShortURLService {
    
    /** *  Estimated number of values to put  */

    private Integer size=10_0000;
    /** *  Expected miscalculation rate  */
    private Double error=0.01;
    /** *  Instantiate the bloom filter  */
    private BloomFilter<String> shortUrlCodeFilter = BloomFilter.create(Funnels.stringFunnel(Charset.defaultCharset()), size, error);

    /** *  Put the short URL code  * * @param code */
    public void put(String code) {
    
        shortUrlCodeFilter.put(code);
    }

    /** *  Determine whether the bloom filter contains this element  * * @param code  Short URL code  * @return */
    public boolean contain(String code) {
    
        return shortUrlCodeFilter.mightContain(code);
    }
}

1.4 302 and 301

Because the system needs to Long link Pass in the background Request forwarding Forward it by , So it uses Request forwarding protocol . There are two types of request forwarding , One is 301, The other is 302, What's the difference between them ？
1）301 permanent Redirect
2）302 temporary Redirect
Because we need to do Data statistics , That is, records Short link Of Traffic volume , So we need to use 302 agreement .

Two 、 The system design

2.1 Database design

The design of this system is a little Simple Short chain generation system , So the design of the table is not complicated , The obligation is simple , The main table is Short link information table , Used to store generated short links .

2.1.1 ER chart

Just 4 A watch ： User table 、 Statistical table 、 Short chain grouping table 、 Short link information table .

• One user You can create multiple groups
• One Group There can be multiple short link messages
• One Short link There can be multiple statistics （ Daily visits ）

2.1.1 Table design

️1. User table

Used to save the registration User information

️2. Short chain information table

Used to record the generated Short link information .

️3. Short chain grouping table

Sign in Users can create new short links grouping , Every successful user registration will have a Default grouping And At least one group .

️4. Short chain access statistics

Used to record Each short link Daily users Traffic volume

3、 ... and 、 Detailed design of the system

3.1 Short chain generation

3.1.1 principle

The process of short link generation is probably , The user enters a Long link , Request short chain generation method , First, the method needs to be Lock , Prevent multiple threads, multiple users At the same time, request the method appear Multithreading problem , After the long chain passes through the short chain generator, a 6 Number of codes , Then go to The bloon filter To determine whether it already exists , If it already exists, it needs to Recall generator Generate , The next step is to take this 6 Save bit code to database and The bloon filter .

3.1.2 flow chart

3.2 Short chain visit website

3.2.1 principle

The short chain access process is roughly ： The user enters a Short link , System analysis Give the last 6 Bit code , Then use this code to The bloon filter Query exists , If There is Just go to cache Get long chain , If caching non-existent Go again if you like database Inquire about , And parse the queried short chain Refresh to cache , Next forward by request , Will get Long link adopt 302 agreement forwarded ; If the bloom filter does not exist 6 Bit code It's direct Throw an exception .

3.2.2 flow chart

Four 、 Conclusion

The application scenarios of short links are very extensive , So when faced with such obligations , Designing a reliable short link becomes very important , Because this system is the first version designed and developed by myself , The design is also very simple , There may be some problems , Welcome to leave a message in the comment area to point out , You can also ask questions and suggestions for modification , thank ！.

5、 ... and 、 Open source address

The source code of the system has been uploaded to the code cloud （Gitee）, Welcome to read , If it helps , Please also order star️.
A commercial version is being developed …

Back end	front end
https://gitee.com/BreezAm/TZ-SHORTURL	https://gitee.com/BreezAm/shorturl-site

Online address ：http://url.breez.work