torch_ Geometric learning 1, messagepassing

Installed version

# for windows10
pip install torch==1.2.0		# 10.0 cuda  Single card GTX1660
pip install torch_geometric==1.4.1
pip install torch_sparse==0.4.4
pip install torch_scatter==1.4.0
pip install torch_cluster==1.4.5

torch_geometric

debug Pattern

with torch_geometric.debug():
	out = model(data.x,data.edge_index)

torch_geometric.nn

1. The messaging MessagePassing

$${\mathbf{x}}_i^{(k)}={\gamma }^{(k)}\left({\mathbf{x}}_i^{(k-1)},{\square }_{j\in \mathcal{N}(i)}{\varphi }^{(k)}\left({\mathbf{x}}_i^{(k-1)},{\mathbf{x}}_j^{(k-1)},{\mathbf{e}}_{j,i}\right)\right),$$

The convolution operator is extended to irregular fields , It can usually be expressed as a neighborhood aggregation , Or the process of messaging .

• ${\mathbf{x}}_i^{(k-1)}$ Representation node $i$ In the $k-1$ Point characteristics of layer ;
• ${\mathbf{e}}_{j,i}$ Indication point $j$ point-to-point $i$ Edge features of , Optional ;
• $\square$ Represents a differentiable 、 Permutation invariance function , for example sum、mean、max;
• $\gamma$ and $\varphi$ Represents a differentiable function , for example MLPs

PyG Provides messageppassing Base class , It helps to create this kind of message passing graph neural network by automatically processing message propagation . The user only needs to define the function $\varphi$, namely message(),$\gamma$ namely update(), And the messaging scheme to be used , namely aggr=“add”, aggr="mean" or aggr=“max”.

class MessagePassing (aggr='add',flow='source_to_target', node_dim=0)

• aggr = (“add”、“mean” or “max”), Aggregation mode ;

• flow = (“source_to_target” or “target_to_source”), The flow direction of messaging ;

• node_dim, Indicates which axis to propagate along .

Correlation function ：

• MessagePassing.propatage(edge_index, size = None)
• MessagePassing.message(...)
• MessagePassing.update(aggr_out,** **...)
• MessagePassing.update(aggr_out,** **...)

Will be applied to GCN On the floor ,GCN The mathematical definition of layer is as follows ：
x i ( k ) = ∑ j ∈ N ( i ) ∪ { i } 1 deg ⁡ ( i ) ⋅ deg ⁡ ( j ) ⋅ ( Θ ⋅ x j ( k − 1 ) ) , \mathbf{x}_i^{(k)} = \sum_{j \in \mathcal{N}(i) \cup \{ i \}} \frac{1}{\sqrt{\deg(i)} \cdot \sqrt{\deg(j)}} \cdot \left( \mathbf{\Theta} \cdot \mathbf{x}_j^{(k-1)} \right), xi(k)​=j∈N(i)∪{ i}∑​deg(i) ​⋅deg(j) ​1​⋅(Θ⋅xj(k−1)​),
among , Firstly, the characteristics of adjacent nodes are analyzed , Through the weight matrix $\theta$ To transform ; Then through its degree , Normalize ; Finally, sum it up . The formula can be divided into the following steps ：

1. Add self-loops to the adjacency matrix.
2. Linear transformation of node characteristic matrix .
3. Calculate the normalization coefficient .
4. Node feature normalization .
5. Sum all adjacent node features .

The first three steps are calculated before message delivery , The last two steps pass MessagePassing Class implementation

import torch
from torch_geometric.nn import MessagePassing
from torch_geometric.utils import add_self_loops, degree

class GCNConv(MessagePassing):
    def __init__(self, in_channels, out_channels):
        super().__init__(aggr='add')  # "Add" aggregation (Step 5).
        self.lin = torch.nn.Linear(in_channels, out_channels)

    def forward(self, x, edge_index):
        # x Of shape by  [N, in_channels]
        # edge_index Of shape by  [2, E]

        # Step 1: Add self-loops to the adjacency matrix.
        edge_index, _ = add_self_loops(edge_index, num_nodes=x.size(0))

        # Step 2:  Linear transformation of node characteristic matrix 
        x = self.lin(x)

        # Step 3:  Normalization operation 
        row, col = edge_index
        deg = degree(col, x.size(0), dtype=x.dtype)
        deg_inv_sqrt = deg.pow(-0.5)
        deg_inv_sqrt[deg_inv_sqrt == float('inf')] = 0
        norm = deg_inv_sqrt[row] * deg_inv_sqrt[col]

        # Step 4-5:  Start spreading news 
        return self.propagate(edge_index, x=x, norm=norm)

    def message(self, x_j, norm):
        # x_j Of shape by  [E, out_channels]

        # Step 4:  Node feature normalization 
        return norm.view(-1, 1) * x_j

2. Graph convolution GCN

Source of the paper ： Semi-supervised Classification with Graph Convolutional Networks