Implementation of resnet-34 CNN with kears

Let's start to realize ResNet-34.
One 、 Create a ResidualUnit layer .

from functools import partial
import tensorflow as tf
from tensorflow import keras



DefaultConv2D = partial(keras.layers.Conv2D, kernel_size=3, strides=1,
                        padding="SAME", use_bias=False)

class ResidualUnit(keras.layers.Layer):
    def __init__(self, filters, strides=1, activation="relu", **kwargs):
        super().__init__(**kwargs)
        self.activation = keras.activations.get(activation)
        self.main_layers = [
            DefaultConv2D(filters, strides=strides),
            keras.layers.BatchNormalization(),
            self.activation,
            DefaultConv2D(filters),
            keras.layers.BatchNormalization()]
        self.skip_layers = []
        if strides > 1:
            self.skip_layers = [
                DefaultConv2D(filters, kernel_size=1, strides=strides),
                keras.layers.BatchNormalization()]

    def call(self, inputs):
        Z = inputs
        for layer in self.main_layers:
            Z = layer(Z)
        skip_Z = inputs
        for layer in self.skip_layers:
            skip_Z = layer(skip_Z)
        return self.activation(Z + skip_Z)

From above , In the constructor , We created all the layers we needed ：
Main layer 、 Skip layer （ When the stride is greater than 1 The need when ）.
stay call（） In the method , We pass the input through the main layer 、 Skip layer , Then add the output layer and apply the activation function .

Two 、 use Sequential Model to build ResNet-34.
This model It's actually a Very long Layer sequence .
Now with the above ResidualUnit class , We can divide each residual unit into a layer .

model = keras.models.Sequential()
model.add(DefaultConv2D(64, kernel_size=7, strides=2,
                        input_shape=[224, 224, 3]))
model.add(keras.layers.BatchNormalization())
model.add(keras.layers.Activation("relu"))
model.add(keras.layers.MaxPool2D(pool_size=3, strides=2, padding="SAME"))
prev_filters = 64
for filters in [64] * 3 + [128] * 4 + [256] * 6 + [512] * 3:
    strides = 1 if filters == prev_filters else 2
    model.add(ResidualUnit(filters, strides=strides))
    prev_filters = filters
model.add(keras.layers.GlobalAvgPool2D())
model.add(keras.layers.Flatten())
model.add(keras.layers.Dense(10, activation="softmax"))

There will be ResidualUnit layer Loop added to the model ：
front 3 individual RU（ Residual unit ） have 64 A filter , Then the rest 4 a 128 individual . And so on .
When the number of filters is the same as the previous RU The layers are the same , Set the stride to 1, Otherwise 2.
Then add ResidualUnit , The last update prev_filters.

3、 ... and 、 Don't forget to instantiate the model .

model.summary()

result ：

Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param # 
=================================================================
 conv2d (Conv2D)             (None, 112, 112, 64)      9408      
                                                                 
 batch_normalization (BatchN  (None, 112, 112, 64)     256       
 ormalization)                                                   
                                                                 
 activation (Activation)     (None, 112, 112, 64)      0         
                                                                 
 max_pooling2d (MaxPooling2D  (None, 56, 56, 64)       0         
 )                                                               
                                                                 
 residual_unit (ResidualUnit  (None, 56, 56, 64)       74240     
 )                                                               
                                                                 
 residual_unit_1 (ResidualUn  (None, 56, 56, 64)       74240     
 it)                                                             
                                                                 
 residual_unit_2 (ResidualUn  (None, 56, 56, 64)       74240     
 it)                                                             
                                                                 
 residual_unit_3 (ResidualUn  (None, 28, 28, 128)      230912    
 it)                                                             
                                                                 
 residual_unit_4 (ResidualUn  (None, 28, 28, 128)      295936    
 it)                                                             
                                                                 
 residual_unit_5 (ResidualUn  (None, 28, 28, 128)      295936    
 it)                                                             
                                                                 
 residual_unit_6 (ResidualUn  (None, 28, 28, 128)      295936    
 it)                                                             
                                                                 
 residual_unit_7 (ResidualUn  (None, 14, 14, 256)      920576    
 it)                                                             
                                                                 
 residual_unit_8 (ResidualUn  (None, 14, 14, 256)      1181696   
 it)                                                             
                                                                 
 residual_unit_9 (ResidualUn  (None, 14, 14, 256)      1181696   
 it)                                                             
                                                                 
 residual_unit_10 (ResidualU  (None, 14, 14, 256)      1181696   
 nit)                                                            
                                                                 
 residual_unit_11 (ResidualU  (None, 14, 14, 256)      1181696   
 nit)                                                            
                                                                 
 residual_unit_12 (ResidualU  (None, 14, 14, 256)      1181696   
 nit)                                                            
                                                                 
 residual_unit_13 (ResidualU  (None, 7, 7, 512)        3676160   
 nit)                                                            
                                                                 
 residual_unit_14 (ResidualU  (None, 7, 7, 512)        4722688   
 nit)                                                            
                                                                 
 residual_unit_15 (ResidualU  (None, 7, 7, 512)        4722688   
 nit)                                                            
                                                                 
 global_average_pooling2d (G  (None, 512)              0         
 lobalAveragePooling2D)                                          
                                                                 
 flatten (Flatten)           (None, 512)               0         
                                                                 
 dense (Dense)               (None, 10)                5130      
                                                                 
=================================================================
Total params: 21,306,826
Trainable params: 21,289,802
Non-trainable params: 17,024
_________________________________________________________________

Reference resources ：
Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, 2nd Edition, author ： Aurelien Geron( French ) , also O Reilly published , Book number 978-1-492-03264-9.