What is Batch Normalization
A method in deep neural networks that stabilizes the distribution of values within a layer and helps speed up learning.
Definition
Batch normalization adds a step to the neural network that aligns the feature values on a mini-batch of data. This helps layers receive more stable inputs, speeds up training, and sometimes reduces sensitivity to initial settings. The method is especially known in deep networks for images.
Example
When training a network for image recognition, batch normalization helps the model converge faster and update weights more consistently.
Why it matters
The term is important for understanding practical neural network engineering: quality depends not only on the layers, but also on the stabilization of training.
How it works
During training, the layer takes the mean and spread of the values in the mini-batch, normalizes them, and then applies the scale and offset parameters it learns.
Where it is used
- deep learning
- computer vision
- stabilization of neural network training
Limitations
The method depends on the batch size and the training or output mode. For small batches, alternative normalization methods may be needed.