Compared with the fully connected network FC, CNN is a local connection, that is, the output of a neuron in the previous layer is only connected to the input of several adjacent neurons in the next layer, and the input of a neuron in the next layer only receives the input of the previous layer. The output of several neighboring neurons

Receptive field (field of view)

Convolution kernel

A translation of the input signal has the same translation of the output signal

The properties of the convolution operation itself, the basic properties of linear time-invariant systems

By properly designing the pooling unit and selecting the activation function, CNN can approximately maintain translation invariance.

Example

N K-1

N-K 1

low pass filter

high pass filter

bandpass filter

For a complex signal that contains various frequency components, different filters implemented by different convolution kernels can obtain components of different varying scales in the signal.

The error between the output of the network output layer and the expected response is used to train the output layer network

The BP algorithm back-propagates the output layer error to each previous layer, and trains the convolution kernels of each layer in turn using the back-propagation error.

z=max{0,a}

Convolution operation between input and convolution kernel Activation function operation

Few shared parameters

Divide and conquer input features of different natures

The two-dimensional convolution operation is equivalent to sliding hij in the Xij data array. When amn needs to be calculated, h00 slides to align with Xmn, and then the product term Xm in j hij is calculated and added.

The effective convolution output size is (D1-K1 1)×(D2-K2 1)

The matrix generated by each convolution kernel h through convolution operation

enter

Convolution kernel

output

Pooling that takes the maximum value of the window, that is, selects the maximum value within a small window as the pooling result

Average within window as pooling result

Fixed point value within window as pooling result

size

pooling stride

Add (K-1)/2 zeros to both ends of the input

Add K/2 zeros to one side and (K/2)-1 zeros to the other side.

Convolution operation stage

Detection level (ReLU function)

Pooling (optional)

3D data volume

tensor convolution kernel

convolution plane

Extract different features of the channel dimension

1×1 convolution kernel

Convolution layer convolution calculation

FC layer fully connected calculation activation output

Pooling layer performs pooling

The FC layer is calculated according to the standard BP backpropagation algorithm.

Convolutional layer and pooling layer backpropagation algorithm

The ReLU activation function trains 6 times faster than the tanh activation function

Use deeper layers, smaller convolution kernels, and multiple convolution layers corresponding to one pooling layer.

Obtain better training results by increasing the depth of CNN

A direct increase in the number of layers will bring negative effects

4 parallel branches

Generate output by branching and merging modules

Each branch contains a 1×1 convolution

The purpose is to divide and conquer to reduce parameters and computational complexity

network degradation problem

Residual network characteristics

residual building block

residual grid structure

Dense network characteristics

dense network structure