AI Agents & AutomationFree templatePDF, Excel, Word export

AI Agent Runbook Template

When an AI agent starts giving wrong answers or acting oddly, the person on call needs a clear page to follow, not a design document. This runbook template covers who owns the agent, what it can touch, how to check its health, and exactly how to pause and roll it back.

Fill in

Live preview
Download:

AI Agent Runbook Template

Runbook: Order Status Agent

Service owner: ERP Platform Lead On-call: #erp-ai-oncall, ext. 4410 Runs on: On-prem inference server GPU01, application server APP03 ERP service account: svc_ai_orders (read-only role)

What the agent can do

None. Read-only queries. Drafts email text for planner to send manually.

Daily health checks

Inference endpoint responds within 5 seconds, nightly data refresh completed, error rate below agreed threshold, GPU memory not saturated.

Symptoms and first response

SymptomLikely causeFirst actionEscalate to
Slow or no answersInference server down or overloadedCheck endpoint and GPU memory, restart serviceERP Platform Lead
Answers look wrong or oldStale data refresh or retrieval faultCheck last refresh time, run regression questionsERP Platform Lead
User sees data they should notPermission or retrieval filter faultPause the agent immediately, preserve logsInformation security
Unexpected ERP transactionTool permission or prompt faultPause the agent, review ERP audit logBusiness owner and security

How to pause the agent

Disable the agent in the admin console, then disable the service account in the ERP. Confirm no new queries appear in the ERP audit log.

Rollback

Redeploy the previous container image and prompt configuration from the release tag, then rerun the regression question set.

After an incident

Record the timeline, notify the business owner, preserve logs, and complete an incident postmortem within five working days. Do not re-enable a paused agent until the owner and information security agree.

Approvals

Service owner / Date
Business owner / Date
Information security / Date

Test the pause procedure at least once before go-live and again after any architecture change. A kill switch nobody has tried is not a control.

Review my agent runbook

Optional: an expert checks your pause, rollback and monitoring steps for gaps.

Optional. The tool and every export work without this. No spam.

How to use this template

Fill in one runbook per agent. List the infrastructure it runs on, the ERP service account it uses and every action it can take. Write the pause and rollback steps in plain commands or console clicks, then have someone who did not build the agent follow them in a test. Keep the exported PDF in your IT service management knowledge base and link it from the monitoring alert.

What to monitor

Agents fail quietly more often than loudly. Beyond uptime, watch signals that show quality and safety are intact.

  • Response time and error rate of the inference endpoint
  • Data freshness for any extract or index the agent relies on
  • Share of answers users flag as wrong
  • Number and type of ERP actions taken by the service account
  • GPU memory and queue depth where models run on your own hardware
  • Unusual query patterns, such as bulk exports

Pause before you diagnose

If an agent shows data to the wrong person or performs an unexpected transaction, pause first and diagnose second. Disabling the ERP service account is a reliable backstop because it works regardless of what the agent software does. Preserve prompts, tool calls and logs before restarting anything, because they are the evidence for the postmortem. For read-only agents with low-risk faults, the first action can be a restart or rollback.

Common mistakes

Typical gaps include a service account with broader rights than the agent needs, a pause method that depends on the failing component, and no regression question set to confirm recovery. Also check that the on-call channel is actually monitored out of hours if the agent supports shift work or plant operations.

Frequently Asked Questions

Is a runbook needed for a read-only agent?

Yes, a shorter one. Read-only agents can still expose data, give wrong answers or consume resources. The runbook tells support staff how to check health, restart safely, and escalate a data exposure concern.

What is the safest kill switch?

Disabling the agent's ERP service account is the most dependable, because it removes access at the system of record. Pair it with an application-level switch so you can stop the agent without touching ERP security settings.

How often should we test the runbook?

Before go-live, after major changes to the agent or infrastructure, and at least once a year. Have someone other than the builder run the steps so gaps in the instructions become visible.

Who should be on call?

Usually the team that supports the ERP platform, with escalation to the agent builders and to information security. Name roles and channels in the runbook so it does not depend on one individual.

Netray can help design read-only-by-default agents with an ERP-side kill switch and audit trail, and can review your runbook against that design.

Talk to an ExpertExplore More Free Tools

Related Tools

AI Agents & Automation

AI Agent Incident Postmortem Template

A blameless postmortem form for AI incidents: timeline, impact, root cause, containment and follow-up actions.

AI Agents & Automation

AI Model Card Template for ERP Use Cases

A one-page model card for each AI model or agent in your ERP landscape, covering intended use, data, limits, evaluation and ownership.

AI Agents & Automation

AI Risk Register Worksheet for ERP Projects

An editable risk register with likelihood x impact scores, owners and mitigations, pre-filled with typical ERP AI risks.

AI Agents & Automation

AI Human Review Capacity Calculator

Estimates reviewer hours, headcount and cost per action when a share of an agent's actions need human approval or sampling.

More AI Agents & Automation tools

See all
Back to All Tools