Definition
The autoregressive approach is widely used in language models: the model generates text one fragment at a time, each time taking into account the already created context. Such models are also used in time series, audio, and other serial data.
Beispiel
The language model takes the beginning of the phrase “Today I want to...” and predicts the likely continuation, then uses the new fragment for the next step.
Warum es wichtig ist
The term is important for understanding generative AI: many responses are created not by one instantaneous act, but by successive predictions.
So funktioniert es
The model estimates the probability of the next token or value based on the past sequence. Once the next element is selected, it is added to the context.
Wo es genutzt wird
- text generation
- time series forecast
- audio generation
Einschränkungen
Sequential generation can accumulate errors. Also, the model is limited to the context window and may lose long-distance connections.
FAQ
Why is “Autoregressive Models” useful to know?
The term is important for understanding generative AI: many responses are created not by one instantaneous act, but by successive predictions.