One 、 What is? Numpy The broadcast mechanism of

numpy The broadcasting mechanism is different shape Of ndarray Array operation , Make it the same shape The mechanism of further operation
For example, we often need to zero mean each dimension of a data matrix , You need to subtract the mean value of the dimension from all the data of the dimension , The code is as follows

import numpy as np
#  Simulate one 5*5 Sample matrix of size , Each row is the eigenvector of a sample 
DATA = np.random.permutation(5*5).reshape((5,5))
DATAMEAN = np.mean(DATA, axis=0)
DATA-DATAMEAN
print(DATA.shape)	# (5,5)
print(DATAMEAN.shape) # (1,5)

Although here DATA and DATAMEAN Of shape Dissimilarity , But they can still subtract , This is it. numpy The broadcasting mechanism plays a role .

Two 、Numpy Rules of broadcasting mechanism

The broadcasting mechanism mentioned earlier is to shape Different arrays (ndarray) Designed for computing ,shape The differences are actually divided into two categories ：1、 Different dimensions ;2、shape The numbers are different
The previous is only for the same dimension (DATA and DATAMEAN All are 2 dimension ),shape Different data are verified , Actually dimesion Different can also be calculated , Examples are as follows

b = np.random.permutation(3*3*2).reshape((3,3,2))
print(b.shape)
c = np.array([1,2])
print(b-c)

In the above code ,b yes 3 dimension ,shape yes (3,3,2));c yes 1 dimension ,shape yes (2,), It can also be subtracted normally without reporting an error .
According to my summary ,Numpy The broadcast mechanism of is encountering shape When different arrays are combined , Follow the steps below to broadcast ：
1. Check whether the dimensions are the same , If the dimensions are different , Make up the dimension before the existing dimension
As mentioned earlier c, Its dimension is 1 dimension ,shape yes (2,), The complement dimension is 3 dimension ,shape yes (1,1,2),c from [1,2] Turned into [[[1],[2]]]. Note here that the dimensions of complementation are complemented in the front , That is to say shape become (1,1,2), It doesn't become (2,1,1)
2. After satisfying the same dimension , The dimension length is 1 Copy the data of other axes on the axis of
Still use the above mentioned c, stay c Of shape become (1,1,2) after , Copy No 3 Data of two axes , Make the broadcast complete afterc[x, y, i] = beforec[1, 1, i], among x,y Is any index that satisfies the condition ,i Is an axis that has not been expanded .

3、 ... and 、 Example verification

There are already two code examples , Here is another example that cannot be calculated after broadcasting , You can follow the previous steps , See why the error is reported ：

b = np.random.permutation(3*3*2).reshape((3,2,3))
print(b.shape)
c = np.array([1,2])
print(b-c)

This example is similar to the previous example , But I only changed b Of shape.
You can see ,b Of shape by (3,2,3), and c Of shape by (2,), Add dimensions anyway , Their last dimension is always different , So we can't make up the same shape. So it will report a mistake , It can also be seen from this , When the dimensions are different , The tail of two calculated arrays shape It has to be the same , Otherwise you can't calculate
Here's another example ：

b = np.random.permutation(3*3*2).reshape((3,2,3))
print(b.shape)
c = np.array([1,2,0])
print(b-c[:,None,None])

This example is a little different from the previous example , Use here c[:, None, None] Forced the c Of shape Turned into (3,1,1), At this time b Of shape yes (3,2,3), The broadcasting mechanism will c Broadcast for shape(3,2,3), The matrix after broadcasting is assumed to be afterc, Satisfy afterc[i, x, y] = c[i, 1,1].
It's not just numpy such ,MATLAB Matrix and pytorch Tensors have a similar mechanism , In the same way , But I didn't make an experimental comparison , Not much .
So far, there is nothing to say about the broadcasting mechanism , The above is based on the summary of my experiment , If there is an error , Please point out the communication ！