Building Your AI Incident Runbook
A runbook is a documented, step-by-step response procedure for a specific type of incident. Unlike a general policy, a runbook is operational — it tells the person responding exactly what to do, in what order, and who to involve.
The AI Incident Runbook Structure
Every AI incident runbook should follow a common structure regardless of the incident type:
- ◆Phase 1: Detection & Initial Assessment — Who identifies the incident? What signals trigger the runbook? What is the initial classification?
- ◆Phase 2: Containment — What can be done immediately to limit the impact? Can the affected model be taken offline? Can API access be restricted?
- ◆Phase 3: Investigation — What evidence needs to be collected? Who conducts the technical investigation? What is the chain of custody for evidence?
- ◆Phase 4: Eradication — What is the root cause? How is it fully removed? If the model is compromised, does it need retraining?
- ◆Phase 5: Recovery — How is the system restored to operation? What validation is required before full deployment resumes?
- ◆Phase 6: Post-Incident Review — What happened? What did we miss? What controls would have prevented or earlier detected this incident?
Runbook: Suspected Prompt Injection
- ◆1. Identify the input that triggered concern and preserve it in an audit log
- ◆2. Review recent inputs for similar patterns — check whether this is isolated or part of a campaign
- ◆3. If the injection appears to have caused real-world action (data sent, transaction made), escalate to Class A immediately
- ◆4. Temporarily restrict the input types that enabled the injection (e.g., block document uploads if indirect injection)
- ◆5. Notify security team and relevant business owner
- ◆6. Conduct full review of what the model accessed or did during the session
- ◆7. Implement additional input validation before re-enabling restricted functionality
Runbook: Suspected Model Extraction
- ◆1. Identify the source account/IP generating anomalous query volumes
- ◆2. Immediately rate-limit or block the source
- ◆3. Preserve logs of all queries from that source
- ◆4. Assess the volume of queries — has enough been collected to build a functional replica?
- ◆5. If extraction is likely successful, assess whether the model represents a significant IP asset and initiate legal and commercial response in parallel
- ◆6. Review API access controls and implement tighter rate limits system-wide
In most organisations, AI incident response sits in an uncomfortable gap between the security team, the data team, and the product team. Define ownership explicitly before an incident occurs. The answer is typically a security incident response team with AI-specific expertise drawn from ML engineering — but this needs to be decided and documented in advance.
You don't need a complete runbook library on day one. Start with the most likely incident for your highest-risk AI system. If you have a customer-facing LLM chatbot, start with a prompt injection runbook. If you have a model exposed via public API, start with model extraction. Build from there.
