Definition
Activation functions decide which signal to pass from one layer of the neural network to the next. Without them, the neural network would be reduced to a simple linear model and would poorly describe complex patterns in text, images, sound, or user behavior. Common functions include ReLU, sigmoid, tanh and variations thereof.
Beispiel
In an image recognition model, the activation function helps to identify increasingly complex features: first contours, then shapes, then parts of an object.
Warum es wichtig ist
The term is useful for those who want to understand why neural networks can solve problems where the usual linear formula is not enough.
So funktioniert es
Each neuron receives numbers, applies a function to them, and passes the result on. The choice of function affects the learning speed, the stability of the model, and the quality of the final predictions.
Wo es genutzt wird
- neural network training
- deep learning
- selection of model architecture
Einschränkungen
Some features may cause gradient decay, unstable learning, or a large number of inactive neurons. In practice, the choice depends on the architecture and task.
FAQ
Why is “Activation Functions” useful to know?
The term is useful for those who want to understand why neural networks can solve problems where the usual linear formula is not enough.