Activation Functions

Définition

Activation functions decide which signal to pass from one layer of the neural network to the next. Without them, the neural network would be reduced to a simple linear model and would poorly describe complex patterns in text, images, sound, or user behavior. Common functions include ReLU, sigmoid, tanh and variations thereof.

Exemple

In an image recognition model, the activation function helps to identify increasingly complex features: first contours, then shapes, then parts of an object.

Pourquoi c'est important

The term is useful for those who want to understand why neural networks can solve problems where the usual linear formula is not enough.

Fonctionnement

Each neuron receives numbers, applies a function to them, and passes the result on. The choice of function affects the learning speed, the stability of the model, and the quality of the final predictions.

Où c'est utilisé

• neural network training
• deep learning
• selection of model architecture

Limites

Some features may cause gradient decay, unstable learning, or a large number of inactive neurons. In practice, the choice depends on the architecture and task.

FAQ