Identification of bolt points in aerial photography based on perception

Purpose of this paper ： 

Automatically detect the bolt points in the video , Make it automatically judge the damage , Fall off , Location of rusted bolts , Reduce labor costs .

Technical principle ：

Extract pictures from the original video... Percent 90 As a training set , Per cent 10 As test set , Manually selected and classified as 3 Species category , That is not a bolt , Normal bolt , Damage to bolts , Through transfer learning, convolutional neural network is used for graphic classification training , After training, use the completion model to distinguish all bolt points in the picture , After discrimination , If this point is damage to the bolt, feed back to the original drawing to show .

Pretreatment stage ：

First, extract the pictures in the video , Considering that the number of frames is 50, Every time 50 Frame extraction once makes the picture have a certain difference .

Select and mark the possible damaged bolts manually .    

For how to select the damaged bolt , Except for a few cases , There are also a large number of bolts that can not completely judge whether they are damaged by observing the pictures alone , Considering the difficulty of subsequent identification .

Therefore, the following principles shall be adopted when selecting bolts ：

1. For bolt points with too deep color depth, we think they may be damaged .

2. For points where the color is significantly different from the surrounding bolts , We think it is possible that .

In addition, normal bolt points shall be intercepted manually , Irrelevant point , And classified storage , Finally get the training set

Normal bolt points 100 individual

Irrelevant point         50 individual

Damage bolt points  35 individual

The pixel of a single bolt point is （15*15） about

Carry out gray analysis for a single bolt point

Single bolt point

We can see from the picture that （R,G,B） The distribution is steep , It is not conducive to the distinction and discrimination of graphics .

In this regard, it is necessary to consider gray-scale two pole processing for bolt points , Make its grayscale distinction more obvious , Considering the time reason and expected results （ That is, the definition of the picture itself is too low , Limited promotion effect ）, We still use the original picture for training .

Construction of neural network :

Through access inception, We only need to change the bottleneck layer to train a small part of the data , Can make it have a certain recognition function .

Deep learning process , Because the data set is small , And the pixels are low , We just need to train 400 Time , You can complete the training . The trained model is saved in run In the folder .

According to the image, we can find , The model thinks it's good to learn .

The structure of bolt point neural network

Testing phase ：

Ideas ： adopt openCV Module similarity identifies all bolt points , And get the coordinates of the point , Save in folder , In batch access through the test program , If it is identified as damage to the bolt, the record shall be saved in a new folder , It's best to feed back to the original drawing .

Therefore, through our accurate test atlas , Manually select some points to test the accuracy of the model

Normal point

According to the judgment point 1 Yes 76% The possibility is normal , Although right, this value is not high .

The problem summary ：

1. The definition of aerial captured image is too low .

2. There is too little damage bolt point training data .

3. There is a certain uncertainty in the process of manually selecting damaged bolt points .

4. inceptionV3 Mainly used for static object recognition , The compatibility of bolt points is relatively low , The accuracy of the results will be greatly affected .

5. Due to time , Proficiency and other reasons , It is difficult to build neural network from scratch and train it in a short time .

6. The current model is only an experimental product, and there is still a distance between it and practical application .