Definição
Activation functions decide which signal to pass from one layer of the neural network to the next. Without them, the neural network would be reduced to a simple linear model and would poorly describe complex patterns in text, images, sound, or user behavior. Common functions include ReLU, sigmoid, tanh and variations thereof.
Exemplo
In an image recognition model, the activation function helps to identify increasingly complex features: first contours, then shapes, then parts of an object.
Por que importa
The term is useful for those who want to understand why neural networks can solve problems where the usual linear formula is not enough.
Como funciona
Each neuron receives numbers, applies a function to them, and passes the result on. The choice of function affects the learning speed, the stability of the model, and the quality of the final predictions.
Onde é usado
- neural network training
- deep learning
- selection of model architecture
Limitações
Some features may cause gradient decay, unstable learning, or a large number of inactive neurons. In practice, the choice depends on the architecture and task.
FAQ
Why is “Activation Functions” useful to know?
The term is useful for those who want to understand why neural networks can solve problems where the usual linear formula is not enough.