Tensorflow realizes web face login system

author

git Address ：https://github.com/chenlinzhong/face-login

Face detection MTCNN

file ：fcce_detect.py

model= os.path.abspath(face_comm.get_conf('mtcnn','model'))
class Detect:
    def __init__(self):
        self.detector = MtcnnDetector(model_folder=model, ctx=mx.cpu(0), num_worker=4, accurate_landmark=False)
    def detect_face(self,image):
        img = cv2.imread(image)
        results =self.detector.detect_face(img)
        boxes=[]
        key_points = []
        if results is not None:  
            #box box 
            boxes=results[0]
            # Face 5 A key point 
            points = results[1]
            for i in results[0]:
                faceKeyPoint = []
                for p in points:
                    for i in range(5):
                        faceKeyPoint.append([p[i], p[i + 5]])
                key_points.append(faceKeyPoint)
        return {
    "boxes":boxes,"face_key_point":key_points}

Multi-task convolutional neural network（ Multitask convolutional neural network ）, Face region detection and face key point detection are put together , Its theme framework is similar to cascade. The whole can be divided into P-Net、R-Net、 and O-Net Three layer network structure .

It is 2016 A multi task neural network model for face detection was proposed by Shenzhen Research Institute of Chinese Academy of Sciences in , The model mainly uses three cascaded networks , The idea of candidate box plus classifier , Fast and efficient face detection . The three cascaded networks are used to quickly generate candidate windows P-Net、 For high-precision candidate window filtering selection R-Net And generate the final bounding box and face key points O-Net. And many convolutional neural network models for image processing , The model also uses the image pyramid 、 Border regression 、 Techniques such as non maximum suppression .

principle
Image pyramid
Do... On the picture Resize operation , Scale the original image to different scales , Generate image pyramid . Then the images of different scales are sent to the three sub networks for training , The purpose is to detect faces of different sizes , So as to realize multi-scale target detection .

Three sub network diagrams
P-Net（Proposal Network）
R-Net（Refine Network）
O-Net（Output Network）

Detailed principle

from P-Net To R-Net, And then to the last O-Net, The image input from the network is getting larger and larger , There are more and more convolution channels , The depth of the network （ The layer number ） It's getting deeper and deeper , Therefore, the accuracy of face recognition should be higher and higher .

MTCNN Loss function of

For face recognition , Use the cross entropy cost function directly , For box regression and key point positioning , Use L2 Loss . Finally, multiply the losses of these three parts by their own weights and add them up , Form the final total loss .

1、 Face recognition loss function （cross-entry loss）

2、 The loss function of the regression box （Euclidean loss）

3、 Loss function of key points （Euclidean loss）

MTCNN Blog

Cut face area facenet embedding

Facenet It's a face recognition system developed by Google , The system is a deep convolution neural network trained based on millions of face data , Face images can be embedding（ mapping ） become 128 The eigenvector of the dimension . Characterized by this vector , use knn perhaps svm And other machine learning methods to realize face recognition .Facenet stay LFW The recognition accuracy on the data set is 0.9963

mtcnn The accuracy of face detection method is very high , But there are still some problems .

One , This method cannot identify a slope greater than 45 Degree face and side face . Pictured 6-2 Shown , When the inclination is greater than 45 Deg. , The system cannot detect faces . The cause of this problem may be mtcnn There are no highly inclined face photos in the network training data . There are two solutions ：（1） Rotate the training photos , Before Dan mtcnn Based on the network finetune;（2） After the video frame is read , After rotating at different angles , Pass in separately mtcnn Face detection . The former method needs to process a large amount of face data ; The latter method will affect the speed of real-time detection .
Two 、 It is possible to recognize animal faces as adult faces

Zhihu article

OpenCV The affine transformation of （Affine Transformation）

file ：face_alignment.py

    def align_face(self,opic,faceKeyPoint):
        img = cv2.imread(opic)
        faceKeyPoint = faceKeyPoint[0]

        # According to the two noses and eyes 3 Point alignment 
        eye1 = faceKeyPoint[0]
        eye2 = faceKeyPoint[1]
        noise = faceKeyPoint[2]
        source_point = np.array(
            [eye1, eye2, noise], dtype=np.float32
        )

        eye1_noraml= [int(x) for x in face_comm.get_conf('alignment','left_eye').split(',')]
        eye2_noraml=[int(x) for x in face_comm.get_conf('alignment','right_eye').split(',')]
        noise_normal=[int(x) for x in face_comm.get_conf('alignment','noise').split(',')]
        # Set the standard face model 

        dst_point = np.array(
            [eye1_noraml,
             eye2_noraml,
             noise_normal],
            dtype=np.float32)

        tranform = cv2.getAffineTransform(source_point, dst_point)

        imagesize=tuple([int(x) for x in face_comm.get_conf('alignment','imgsize').split(',')])
        img_new = cv2.warpAffine(img, tranform, imagesize)
        new_image= os.path.abspath(face_comm.get_conf('alignment','aligment_face_dir'))
        new_image= new_image+'/'+'%d_%d.png'%(time.time(),random.randint(0,100))
        if cv2.imwrite(new_image, img_new):
            return new_image
        return None

Geometric transformation of images —— The tensile 、 shrinkage 、 Distortion 、 rotate （stretch,shrink,distortion,rotation）

Affine transformation is a linear transformation in vector space ( Multiply by a matrix ) And add a translation ( Add a vector ), The process of transforming into another vector space . In the case of finite dimensions , Each affine transformation can be represented by a matrix A And a vector b give , It can write A And an additional column b. An affine transformation corresponds to the multiplication of a matrix and a vector , The composition of affine transformation corresponds to ordinary matrix multiplication , Just add an extra row to the bottom of the matrix , This line is all about 0 Except for the one on the far right 1, And you have to add a column vector at the bottom 1.

It is a transformation from two-dimensional coordinates to two-dimensional coordinates

OpenCV Affine transformation is realized by the combination of two functions , That is, the two steps of affine transformation ：

First step ： Get affine mapping matrix （ Two kinds of ）：

The second step ： Affine transformation

OpenCV Learning notes

Affine transformation warpAffine

Expand ： Perspective transformation (perspective transform)

stay 3D In plane , Perspective transformation has its place again . The two transformation principles are similar , The results are similar , Appropriate transformation can be used for different occasions

Face feature index

# Face features are stored in lmdb Format in file (id,vector), So here from lmdb File loading

lmdb_file = self.lmdb_file if os.path.isdir(lmdb_file):
evn = lmdb.open(lmdb_file)
wfp = evn.begin()
annoy = AnnoyIndex(self.f)
for key, value in wfp.cursor():
key = int(key)
value = face_comm.str_to_embed(value)
annoy.add_item(key,value)

annoy.build(self.num_trees)
annoy.save(self.annoy_index_path)

In face recognition, we can't compare everyone's face , Too slow , The feature indexes obtained by the same people are more similar , May adopt KNN Classification algorithm to identify , It is more efficient to use annoy The algorithm creates an index of face features

annoy The goal of the algorithm is to establish a data structure that can find the nearest point of any query point in a short time , Under the condition of accuracy, the search speed is much faster than violent search by sacrificing accuracy .

Annoy search algorithm (Approximate Nearest Neighbors Oh Yeah)

In terms of data scale , If your vector dimension is less than 1000, The number of vectors is in the millions level , Take a quick ride demo, It is recommended to use annoy,annoy It's faster to get started , Support more distances . If there's more data （ Such as billion）, Then it's best to use faiss. In terms of performance ,annoy Than faiss-ivf The performance is slightly worse , however faiss The performance of depends on index.