Definition
Model drift is when an agent's output quality changes without its configuration changing, because the underlying model, an API, or a data source changed instead.
In practice
It is the failure mode most often misdiagnosed as a prompt problem. A model version is retired and traffic moves to its successor, an API renames a field, or a site the agent reads redesigns its markup. The agent is running perfectly and doing something subtly different. Drift is hard to catch because it degrades rather than breaks: the brief still arrives at 7am, it is just no longer worth reading. The only reliable defence is a habit rather than a tool, namely reading a sample of real output regularly, and re-testing deliberately whenever a model or dependency changes.
Example
A weekly market report has run on the same prompt for months. After the provider retires the model it used and routes traffic to a newer one, the report still arrives on time, but it is longer, more hedged, and drops the price comparisons readers relied on. Nothing errored and nothing in the agent changed. The owner only notices when a reader asks where the numbers went.
Common questions about model drift
How do you catch model drift?
Model drift degrades output rather than breaking it, so monitoring for errors will not catch it. Read a sample of real output on a regular schedule, compare it with an example you were happy with, and re-test deliberately whenever a model version, an API, or a data source the agent depends on changes.
Is model drift a prompt problem?
Usually not, though it is often misdiagnosed as one. Model drift means something beneath the agent changed: the model version, an API's response format, or the markup of a page it reads. Rewriting the prompt can compensate, but find what changed first, or the next dependency update will break it again.
Read more
Nearby terms
All 25 termsRun agents without the homework
Qoren runs OpenClaw, Hermes, and Codex agents in managed environments, with the restarts, secrets, schedules, and spend caps handled for you.