Définition
The autoregressive approach is widely used in language models: the model generates text one fragment at a time, each time taking into account the already created context. Such models are also used in time series, audio, and other serial data.
Exemple
The language model takes the beginning of the phrase “Today I want to...” and predicts the likely continuation, then uses the new fragment for the next step.
Pourquoi c'est important
The term is important for understanding generative AI: many responses are created not by one instantaneous act, but by successive predictions.
Fonctionnement
The model estimates the probability of the next token or value based on the past sequence. Once the next element is selected, it is added to the context.
Où c'est utilisé
- text generation
- time series forecast
- audio generation
Limites
Sequential generation can accumulate errors. Also, the model is limited to the context window and may lose long-distance connections.
FAQ
Why is “Autoregressive Models” useful to know?
The term is important for understanding generative AI: many responses are created not by one instantaneous act, but by successive predictions.