AI Agent Incident Postmortem Template
When an AI agent gets something wrong, the useful question is how the system allowed it. This blameless postmortem template captures the timeline, impact, root cause and follow-up actions in a format that quality, security and IT can all use.
Fill in
AI Agent Incident Postmortem Template
AI Incident Postmortem AI-2026-014
Agent or system: Order Status Agent Date detected: 2026-10-12 Severity: Medium: wrong output used internally Reported by: Customer Service Planner
This review is blameless. Its purpose is to understand how the system allowed the event and to reduce the chance of repeats.
Summary
The agent reported a shipment date from a stale extract. A planner quoted the date to a customer. The correct date was three days later.
Timeline
09:10 planner asks agent for ship date 09:12 agent answers from nightly extract 11:40 customer queries the date 11:55 planner finds the conflict in the ERP 12:20 agent paused by ERP platform lead
Impact
One customer received an incorrect date for about two hours. No financial loss.
Root cause and contributing factors
Nightly data refresh job failed silently two days earlier. The agent did not show data freshness, and no alert existed for the failed job.
Containment checklist
| Step | Done | By | Time |
|---|---|---|---|
| Agent paused or access restricted | |||
| Logs, prompts and tool calls preserved | |||
| Affected users and data owners informed | |||
| Security and compliance notified if data was exposed | |||
| Wrong records or external messages corrected |
Corrective and preventive actions
| Action | Owner | Due date | Status |
|---|---|---|---|
| Add data freshness display to every answer | |||
| Alert on failed refresh job | |||
| Add case to regression test set | |||
| Update model card and runbook |
Approvals
Complete within five working days of containment. Link the postmortem to the model card and risk register entries for the agent, and re-test before re-enabling a paused agent.
Review an AI incident with an expert
Optional: an expert reads your postmortem and suggests technical controls to prevent a repeat.
Optional. The tool and every export work without this. No spam.
How to use this template
Open a postmortem for any incident above your severity threshold, and consider short versions for near misses, since they reveal the same weaknesses at no cost. Fill in the summary and timeline first using logs, not memory. Then work back to the root cause and contributing factors, and list actions with owners and due dates. Export the document, attach it to the incident ticket, and track the actions to closure.
Finding root causes in AI incidents
AI incidents usually have layered causes: a data problem, a missing control and a human reliance on the output. Ask why repeatedly, and look across these areas rather than blaming the model alone.
- Data: stale, duplicated or wrongly permissioned sources
- Retrieval: wrong document or query selected
- Model: unsupported inference or invented detail
- Controls: missing review step, freshness display or alert
- Process: users trusting output without checking the source
- Change: recent model, prompt or data source update
Severity and who to tell
Agree severity levels before you need them. Wrong output used internally is a quality matter. Data exposure or an unintended ERP transaction is a security and operations matter and should bring in information security immediately. Anything with customer, regulatory or safety impact needs senior management and, depending on your obligations, may trigger notification duties under contracts or privacy rules. Check with legal and compliance on what applies to you.
Common mistakes
Postmortems lose value when they stop at human error, when actions have no owners, and when they are never fed back into testing. Add each incident to the regression question set, update the model card and runbook, and review open actions at the same meeting where you review the risk register.
Frequently Asked Questions
What counts as an AI incident?
Any event where an AI system caused or nearly caused a wrong decision, data exposure, unintended system change, or breach of policy. Include near misses, because they show weaknesses before real harm occurs.
Why blameless?
People report problems quickly when they are not punished for them. A blameless format focuses on system and process weaknesses, such as missing alerts or controls, which are fixable, instead of individual mistakes.
When should an agent be re-enabled?
After containment is confirmed, the cause is understood or controlled, the case is added to the test set, and the service owner and information security agree. Record the decision and date in the postmortem.
Do we need to notify customers or regulators?
It depends on contracts, privacy laws and sector rules. Involve legal and compliance early for any data exposure or external impact, and document the decision about notification in the postmortem.
If incidents keep tracing back to stale data or missing audit trails between your ERP and AI tools, Netray can review the integration design and logging with you.
Related Tools
AI Agent Runbook Template
A runbook for the person on call when an ERP-connected AI agent misbehaves: how to check it, pause it and roll back.
AI Agents & AutomationAI Risk Register Worksheet for ERP Projects
An editable risk register with likelihood x impact scores, owners and mitigations, pre-filled with typical ERP AI risks.
AI Agents & AutomationAI Model Card Template for ERP Use Cases
A one-page model card for each AI model or agent in your ERP landscape, covering intended use, data, limits, evaluation and ownership.
AI Agents & AutomationAI Human Review Capacity Calculator
Estimates reviewer hours, headcount and cost per action when a share of an agent's actions need human approval or sampling.
More AI Agents & Automation tools
See all- Business Process Automation ROI Calculator
- AI Agent Use Case Prioritizer
- Document Processing Automation Savings Calculator
- AP Invoice Automation ROI Calculator
- Order Entry Automation Savings Calculator
- Quote-to-Cash Cycle Time Calculator
- Manufacturing AI Readiness Assessment
- RPA vs AI Agent Selector
- ERP AI Maturity Assessment