1 Introduce torch.nn.Module

1. It is the of all neural networks base class
2. All the networks we write should inherit this class

• A simple Model
  

2 add_module（name, module） Method

• To the current module , Add submodule

add_module(name, module)
 Method : add_module(name, module)
    Adds a child module to the current module.
     Add a sub module to the current module .
    The module can be accessed as an attribute using the given name.
     By a given name , We can access the module by accessing properties .
    Parameters   Parameters 
            name (string) – name of the child module. The child 
            module can be accessed from this module using the 
            given name
			name ( character string ) –  The name of the submodule .  Use the given in the current module name Just 
			 You can access sub modules .
            module (Module) – child module to be added to the module.
            module ( modular ) –  Sub modules to be added to the current module .

2.1 Code implementation

• It can run directly , This code was moved over , Paste the connection of the original post ：

https://blog.csdn.net/m0_46653437/article/details/112649366?ops_request_misc=%257B%2522request%255Fid%2522%253A%2522165062788416780366588245%2522%252C%2522scm%2522%253A%252220140713.130102334.pc%255Fall.%2522%257D&request_id=165062788416780366588245&biz_id=0&utm_medium=distribute.pc_search_result.none-task-blog-2_allfirst_rank_ecpm_v1~rank_v31_ecpm-1-112649366.142^v9control,157^v4control&utm_term=add_module%EF%BC%88name%2C+module%EF%BC%89&spm=1018.2226.3001.4187

import torch
import torch.nn as nn

torch.manual_seed(seed=20200910)


class Model(torch.nn.Module):
    def __init__(self):
        super(Model,self).__init__()
        self.conv1 = torch.nn.Sequential(  #  Input torch.Size([64, 1, 28, 28])
                torch.nn.Conv2d(1, 64, kernel_size=3, stride=1, padding=1),
                torch.nn.ReLU(),  #  Output torch.Size([64, 64, 28, 28])
                torch.nn.Conv2d(64, 128, kernel_size=3, stride=1, padding=1),  #  Output torch.Size([64, 128, 28, 28])
                torch.nn.ReLU(),
                torch.nn.MaxPool2d(stride=2, kernel_size=2)  #  Output torch.Size([64, 128, 14, 14])
        )

        self.dense = torch.nn.Sequential(  #  Input torch.Size([64, 14*14*128])
                    torch.nn.Linear(14*14*128, 1024),  #  Output torch.Size([64, 1024])
                    torch.nn.ReLU(),
                    torch.nn.Dropout(p=0.5),
                    torch.nn.Linear(1024, 10)  #  Output torch.Size([64, 10])
        )
        self.layer4cxq1 = torch.nn.Conv2d(2, 33, 4, 4)
        self.layer4cxq2 = torch.nn.ReLU()
        self.layer4cxq3 = torch.nn.MaxPool2d(stride=2, kernel_size=2)
        self.layer4cxq4 = torch.nn.Linear(14*14*128, 1024)
        self.layer4cxq5 = torch.nn.Dropout(p=0.8)
        self.attribute4cxq = nn.Parameter(torch.tensor(20200910.0))
        self.attribute4lzq = nn.Parameter(torch.tensor([2.0, 3.0, 4.0, 5.0]))
        self.attribute4hh = nn.Parameter(torch.randn(3, 4, 5, 6))
        self.attribute4wyf = nn.Parameter(torch.randn(7, 8, 9, 10))

    def forward(self, x):  # torch.Size([64, 1, 28, 28])
        x = self.conv1(x)  #  Output torch.Size([64, 128, 14, 14])
        x = x.view(-1, 14*14*128)  # torch.Size([64, 14*14*128])
        x = self.dense(x)  #  Output torch.Size([64, 10])
        return x


print('cuda(GPU) Is it available :', torch.cuda.is_available())
print('torch Version of :', torch.__version__)

model = Model()  #.cuda()

print(" test model (CPU)".center(100, "-"))
print(type(model))

print("torch.nn.Module.add_module(name, module) Before method call ".center(100, "-"))
for name, child in model.named_modules():
    print(' The name of the module is :', name, '### The module itself is :', child)


print("torch.nn.Module.add_module(name, module) After method call ".center(100,"-"))
model.add_module('JUJU', torch.nn.Conv2d(38, 38, 38, 38))
for name, child in model.named_modules():
    print(' The name of the module is :', name, '### The module itself is :', child)

cuda(GPU) Is it available : False
torch Version of : 1.11.0+cpu
--------------------------------------------- test model (CPU)----------------------------------------------
<class '__main__.Model'>
---------------------------torch.nn.Module.add_module(name, module) Before method call ---------------------------
 The name of the module is :  ### The module itself is : Model(
  (conv1): Sequential(
    (0): Conv2d(1, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): ReLU()
    (2): Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (3): ReLU()
    (4): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  )
  (dense): Sequential(
    (0): Linear(in_features=25088, out_features=1024, bias=True)
    (1): ReLU()
    (2): Dropout(p=0.5, inplace=False)
    (3): Linear(in_features=1024, out_features=10, bias=True)
  )
  (layer4cxq1): Conv2d(2, 33, kernel_size=(4, 4), stride=(4, 4))
  (layer4cxq2): ReLU()
  (layer4cxq3): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  (layer4cxq4): Linear(in_features=25088, out_features=1024, bias=True)
  (layer4cxq5): Dropout(p=0.8, inplace=False)
)
 The name of the module is : conv1 ### The module itself is : Sequential(
  (0): Conv2d(1, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (1): ReLU()
  (2): Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (3): ReLU()
  (4): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
)
 The name of the module is : conv1.0 ### The module itself is : Conv2d(1, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
 The name of the module is : conv1.1 ### The module itself is : ReLU()
 The name of the module is : conv1.2 ### The module itself is : Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
 The name of the module is : conv1.3 ### The module itself is : ReLU()
 The name of the module is : conv1.4 ### The module itself is : MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
 The name of the module is : dense ### The module itself is : Sequential(
  (0): Linear(in_features=25088, out_features=1024, bias=True)
  (1): ReLU()
  (2): Dropout(p=0.5, inplace=False)
  (3): Linear(in_features=1024, out_features=10, bias=True)
)
 The name of the module is : dense.0 ### The module itself is : Linear(in_features=25088, out_features=1024, bias=True)
 The name of the module is : dense.1 ### The module itself is : ReLU()
 The name of the module is : dense.2 ### The module itself is : Dropout(p=0.5, inplace=False)
 The name of the module is : dense.3 ### The module itself is : Linear(in_features=1024, out_features=10, bias=True)
 The name of the module is : layer4cxq1 ### The module itself is : Conv2d(2, 33, kernel_size=(4, 4), stride=(4, 4))
 The name of the module is : layer4cxq2 ### The module itself is : ReLU()
 The name of the module is : layer4cxq3 ### The module itself is : MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
 The name of the module is : layer4cxq4 ### The module itself is : Linear(in_features=25088, out_features=1024, bias=True)
 The name of the module is : layer4cxq5 ### The module itself is : Dropout(p=0.8, inplace=False)
---------------------------torch.nn.Module.add_module(name, module) After method call ---------------------------
 The name of the module is :  ### The module itself is : Model(
  (conv1): Sequential(
    (0): Conv2d(1, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): ReLU()
    (2): Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (3): ReLU()
    (4): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  )
  (dense): Sequential(
    (0): Linear(in_features=25088, out_features=1024, bias=True)
    (1): ReLU()
    (2): Dropout(p=0.5, inplace=False)
    (3): Linear(in_features=1024, out_features=10, bias=True)
  )
  (layer4cxq1): Conv2d(2, 33, kernel_size=(4, 4), stride=(4, 4))
  (layer4cxq2): ReLU()
  (layer4cxq3): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  (layer4cxq4): Linear(in_features=25088, out_features=1024, bias=True)
  (layer4cxq5): Dropout(p=0.8, inplace=False)
  (JUJU): Conv2d(38, 38, kernel_size=(38, 38), stride=(38, 38))
)
 The name of the module is : conv1 ### The module itself is : Sequential(
  (0): Conv2d(1, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (1): ReLU()
  (2): Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (3): ReLU()
  (4): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
)
 The name of the module is : conv1.0 ### The module itself is : Conv2d(1, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
 The name of the module is : conv1.1 ### The module itself is : ReLU()
 The name of the module is : conv1.2 ### The module itself is : Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
 The name of the module is : conv1.3 ### The module itself is : ReLU()
 The name of the module is : conv1.4 ### The module itself is : MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
 The name of the module is : dense ### The module itself is : Sequential(
  (0): Linear(in_features=25088, out_features=1024, bias=True)
  (1): ReLU()
  (2): Dropout(p=0.5, inplace=False)
  (3): Linear(in_features=1024, out_features=10, bias=True)
)
 The name of the module is : dense.0 ### The module itself is : Linear(in_features=25088, out_features=1024, bias=True)
 The name of the module is : dense.1 ### The module itself is : ReLU()
 The name of the module is : dense.2 ### The module itself is : Dropout(p=0.5, inplace=False)
 The name of the module is : dense.3 ### The module itself is : Linear(in_features=1024, out_features=10, bias=True)
 The name of the module is : layer4cxq1 ### The module itself is : Conv2d(2, 33, kernel_size=(4, 4), stride=(4, 4))
 The name of the module is : layer4cxq2 ### The module itself is : ReLU()
 The name of the module is : layer4cxq3 ### The module itself is : MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
 The name of the module is : layer4cxq4 ### The module itself is : Linear(in_features=25088, out_features=1024, bias=True)
 The name of the module is : layer4cxq5 ### The module itself is : Dropout(p=0.8, inplace=False)
 The name of the module is : JUJU ### The module itself is : Conv2d(38, 38, kernel_size=(38, 38), stride=(38, 38))

Process finished with exit code 0

2.2 summary

• From the last line of output you can see , there add_module It was actually implemented , It's directly added to init After the function , But not in forward Function ,forward The network in the function is the neural network used in the calculation of the later model .

3 apply(fn) Method

• Used to randomly initialize a parameter

4 bfloat16() Method

• Convert all floating-point numbers to bfloat16 Type of

5 parameters()

• torch.nn.Parameter It is inherited from torch.Tensor Subclasses of , Its main function is as nn.

• Module Use the trainable parameters in . It is associated with torch.Tensor The difference is that nn.Parameter Will automatically be considered module The trainable parameters of , That is to add to parameter() This iterator goes to ; and module Central African nn.Parameter() Ordinary tensor Yes, no parameter Medium .

• nn.Parameter Of requires_grad The default value of the property is True, It's something that can be trained , This is related to torh.Tensor The default value of the object is the opposite .

• stay nn.Module Class ,pytorch Is also used nn.Parameter To every one of them module Parameters of .