AI agent maintenance checklist

The checklist at a glance

Daily: did it run, and what did it cost

The whole daily check is two questions, and with run records in front of you it takes under a minute. Without them it takes ten and you skip it, which is how agents end up dead for a week.

• Did every scheduled run fire? A missing run matters more than a failed one, because a failure at least tells you something.
• Is yesterday's spend in the normal range? A step change is the earliest sign of a retry loop.
• Anything in the error log that is not the usual flaky-network noise?

Weekly: is the output still good

The check that catches quality drift, and the one no tool can do for you. Read a sample the way a reader would, not the way its author would.

• Read two or three actual outputs end to end. Would you act on them?
• Look for repetition: the same items resurfacing every day usually means a filter or a date window broke.
• Scan errors for anything that appeared more than twice. Once is noise, three times is a pattern.
• Check anything the agent touches that you did not read: files written, messages sent, tickets created.

Monthly: the host and the credentials

Mostly infrastructure work, and mostly what a managed platform removes. If you self-host, none of it is optional.

• Apply operating system and dependency security updates, then confirm the agent came back up afterwards.
• Review which credentials exist, what each can do, and whether any are broader in scope than the agent needs.
• Check the month's total spend against the month before, per agent rather than in aggregate.
• Confirm your backups, or confirm honestly that you have none and that this is a decision rather than an oversight.

Quarterly: rotation, drift, and pruning

The slow-moving work. Easy to defer indefinitely, which is exactly why it belongs on a calendar rather than in your memory.

• Rotate provider and integration keys, and remove any that no longer correspond to a running agent.
• Re-test against current models. Deprecations and version changes shift behaviour without shifting your prompt.
• Re-test integrations whose APIs or markup may have changed since you last looked.
• Prune. An agent whose output nobody has read in three months should be turned off, not maintained.

What a managed platform removes, honestly

Not everything, and it is worth knowing which is which before you decide. Qoren handles host patching, restarts, health monitoring, encrypted secret storage, run records, and a hard spend cap, so most of the monthly list and half of the daily one stop being yours.

• Removed: host patching, restart policies, uptime alerting, run recording, spend enforcement, and secret storage.
• Reduced: credential rotation, which becomes one edit in one place rather than a hunt through files and shell history.
• Unchanged: reading the output, judging whether it is good, and re-testing after a model changes. That is the judgment work, and no host can take it.