module class

pytorch In fact, there is generally no particularly obvious Layer and Module The difference between , Whether it's a custom layer 、 Custom blocks 、 Custom model , It's all through inheritance Module Class , Actually Sequential Class is also inheritance Module class .

module The definition of a class ：

class Module(object):
	def __init__(self):
	def forward(self, *input):
	
	def add_module(self,name,module):
	def cuda(self,device=None):
	def cpu(self):
	def __call__(self, *input, **kwargs):
	def parameters(self, recurse=True):
	def named_parameters(self, prefix='', recurse=True):
	def children(self):
	def named_children(self):
	def modules(self):
	def named_modules(self,memo=None,prefix=''):
	def train(self,mode=True):
	def eval(self):
	def zero_grad(self):
	def __repr__(self):
	def __dir__(self):

When we define our own network , Need to inherit nn.Module class , And re implement the constructor __init__ Constructors and forward These two methods .
Attention skills ：
（1） Generally, the layers with learnable parameters in the network （ Such as full connection layer , Convolution layer, etc ） Put it in the constructor __init__() in , Of course, I can also put layers without parameters inside
（2） Generally, layers that do not have learnable parameters （ Such as ReLU, dropout, BatchNormalnation layer ） Can be placed in the constructor , It can also be left out of the constructor , If you don't put it in the constructor __init__ Inside , It's in forward You can use nn.functional Instead of
（3）forward Methods must be rewritten , It is the function of implementing the model , The core of realizing the connection relationship between various layers

Be careful ： As long as module in __init__ Defined layer , Even if forward Not used in , When saving the model , The network layer will also be saved , So if you don't need something, don't put it in __init__

torch.nn.Module Multiple implementations of classes

Method 1： adopt Sequential Layer to layer , That is, several layers are packed together as a large layer

import torch.nn as nn
from collections import OrderedDict
class MyNet(nn.Module):
	def __init__(self):
		super(MyNet, self).__init__()
		self.conv_block = nn.Sequential(
			nn.Conv2d(3, 32, 3,1,1),
			nn.ReLU(),
			nn.MaxPool2d(2)
		)
		self.dense_block = nn.Sequential(
			nn.Linear(32*3*3, 128),
			nn.ReLU(),
			nn.Linear(128,10)
		)
	def forward(self,x):
		conv_out = self.conv_block(x)
		res = conv_out.view(conv_out.size(0),-1)
		out = self.dense_block(res)
		return out

model = MyNet()
print(model)

Here, in every package , Each layer has no name , The default in accordance with the 0,1,2,3,4 Sort .
Method 2

import torch.nn as nn
from collections import OrderedDict
class MyNet(nn.Module):
    def __init__(self):
        super(MyNet, self).__init__()
        self.conv_block = nn.Sequential(
            OrderedDict(
                [
                    ("conv1", nn.Conv2d(3, 32, 3, 1, 1)),
                    ("relu1", nn.ReLU()),
                    ("pool", nn.MaxPool2d(2))
                ]
            ))
 
        self.dense_block = nn.Sequential(
            OrderedDict([
                ("dense1", nn.Linear(32 * 3 * 3, 128)),
                ("relu2", nn.ReLU()),
                ("dense2", nn.Linear(128, 10))
            ])
        )
 
    def forward(self, x):
        conv_out = self.conv_block(x)
        res = conv_out.view(conv_out.size(0), -1)
        out = self.dense_block(res)
        return out
 
model = MyNet()
print(model)

Method 3

import torch.nn as nn
from collections import OrderedDict
class MyNet(nn.Module):
    def __init__(self):
        super(MyNet, self).__init__()
        self.conv_block=torch.nn.Sequential()
        self.conv_block.add_module("conv1",torch.nn.Conv2d(3, 32, 3, 1, 1))
        self.conv_block.add_module("relu1",torch.nn.ReLU())
        self.conv_block.add_module("pool1",torch.nn.MaxPool2d(2))
 
        self.dense_block = torch.nn.Sequential()
        self.dense_block.add_module("dense1",torch.nn.Linear(32 * 3 * 3, 128))
        self.dense_block.add_module("relu2",torch.nn.ReLU())
        self.dense_block.add_module("dense2",torch.nn.Linear(128, 10))
 
    def forward(self, x):
        conv_out = self.conv_block(x)
        res = conv_out.view(conv_out.size(0), -1)
        out = self.dense_block(res)
        return out
 
model = MyNet()
print(model)

The above method 2 And methods 3, In each package , Each layer has a name .
Particular attention ：Sequential Although class inherits from Module class , The two have similar parts , But there are many different parts , Focus on ：

Sequential Class implements integer indexing , Therefore, it can be used model[index] Get a layer in this way , however Module Class does not implement integer indexing , It is not possible to obtain the layer by integer index , But it provides several main methods , as follows ：
The details can be found in My blog

children and modules Differences between

Be careful pytorch Whether it's a model 、 layer 、 Activation function 、 The loss function can be regarded as Module Development of , therefore modules and named_modules Will iterate layer by layer , from the shallower to the deeper , Add each custom block block、 then block Every layer inside is regarded as module To iterate , and children It's more intuitive , It means the so-called “ children ”, So there are no layers of depth

Reference resources ：
https://zhuanlan.zhihu.com/p/156127643