Run a Tabletop Exercise for AI Safety Incidents
This guide walks you through designing and facilitating a tabletop exercise (TTX) focused on AI safety incidents. You’ll learn how to simulate a realistic incident, test your team’s response plans, and identify gaps—all without deploying any actual AI systems. The steps are practical and adaptable for any organization that uses AI, from startups to enterprises.
What You’ll Need
- A facilitator (could be internal or external) who understands AI basics and incident response.
- A participant group of 5–12 people, representing roles like engineering, product, legal, communications, and executive leadership.
- An incident scenario script (you’ll draft this in Step 1).
- A whiteboard or shared digital document (e.g., Google Docs, Miro) for capturing notes.
- Printed or digital copies of your existing incident response plan, AI usage policies, and escalation procedures.
- At least 90 minutes of uninterrupted time (2 hours is ideal).
- Optional: a timer or timekeeper to track each phase.
Step-by-Step
- Define your objectives. Before the exercise, write down 2–3 specific goals. For example: “Test our ability to detect a model bias issue within 30 minutes� or “Evaluate our communication flow to the CEO during a safety incident.� These goals will shape the scenario and keep the exercise focused.
- Draft a realistic scenario. Create a one-page description of an AI incident that could plausibly happen in your organization. Include: the AI system involved (e.g., a customer-facing chatbot), the trigger (e.g., users report offensive outputs), and a few evolving complications (e.g., media picks it up, regulators ask questions). Avoid making it too extreme—realistic scenarios drive better discussions.
- Assign roles and participants. Assign each participant a role they would actually play in a real incident (e.g., Incident Commander, Legal Lead, Communications Lead, Engineering Lead). If you have more people than roles, let some be “observers� who take notes and give feedback at the end.
- Brief participants (15 minutes). At the start of the session, explain the rules: this is a discussion exercise, not a live simulation. There are no “right� answers; the goal is to surface strengths and gaps. Hand out the scenario and the current incident response plan (if you have one).
- Run the exercise in phases (60–75 minutes). Divide the time into 3–4 phases, each introducing a new piece of information (e.g., “First reports come in,� “A journalist asks for comment,� “The model is still misbehaving�). For each phase, read the inject aloud, then give participants 10–15 minutes to discuss and decide their actions. The facilitator should ask probing questions like “What do you do first?� and “Who needs to be informed?�
- Capture decisions and gaps. As participants discuss, the facilitator or a scribe writes down key decisions, open questions, and any missing procedures on the whiteboard. Note where the team hesitates or disagrees—these are valuable findings.
- Conduct a hotwash debrief (15–20 minutes). After the last phase, lead a structured discussion: What went well? What was unclear? What would you change in your real plan? Encourage honest feedback. Capture the top 3–5 action items.
- Write and distribute a summary report. Within a week, send a concise report to all participants and leadership. Include: objectives, scenario summary, key findings, and prioritized action items with owners and deadlines.
Common Mistakes
- Making the scenario too technical. If only engineers can follow it, other participants disengage. Keep the scenario understandable to all roles.
- Letting one person dominate. The facilitator must ensure every voice is heard, especially from non-technical roles.
- Skipping the debrief. The exercise’s value comes from reflection. Without a structured debrief, you lose the learning.
- Not tying findings to real plans. If you don’t update your incident response plan based on the exercise, it was just a fun meeting.
- Over-scripting. If you plan every minute, you leave no room for organic discussion. Let the conversation breathe.
Tips
- Start small. If this is your first TTX, run a simple scenario with a small group before tackling a complex one.
- Use a neutral facilitator. An external facilitator can ask uncomfortable questions without bias.
- Encourage “what if� thinking. Ask participants to consider worst-case outcomes, even if they seem unlikely.
- Record the session (with consent). Audio or video recording helps you capture details you might miss in notes.
- Make it a regular habit. Run a TTX every 6–12 months, or after any major change to your AI systems.
Troubleshooting
- Participants go off-topic. Gently steer them back by saying, “Let’s hold that thought and return to the incident at hand.� If they keep drifting, note the tangent as an action item for later.
- The scenario falls flat. If participants aren’t engaged, introduce a new inject (e.g., “The bug is now affecting 10,000 users�) to raise stakes.
- People don’t know the plan. If participants are unfamiliar with your incident response plan, pause the exercise and give them 10 minutes to read it. This is a finding in itself.
- Time runs out. If you’re short on time, skip the last inject and go straight to the debrief. The debrief is more important than finishing all phases.
- Conflict arises. If two departments disagree on ownership, let them discuss briefly, then note it as an open question for leadership to resolve.
Frequently Asked Questions
How long does a tabletop exercise take?
A basic TTX can be done in 90 minutes, but 2–3 hours allows for deeper discussion. For complex AI systems, you may want a half-day.
Do we need a real AI incident to run one?
No. The scenario is fictional but realistic. The goal is to practice your response, not to recreate a real event.
Who should participate?
Include anyone who would be involved in a real incident: engineers, product managers, legal, PR, and executives. Cross-functional participation is key.
How often should we run these exercises?
At least once a year, or whenever you deploy a new AI system or change your incident response procedures.
What if we don’t have an incident response plan yet?
Run the TTX anyway. It will highlight the need for a plan and help you draft one based on the gaps you discover.
Can we use a tabletop exercise for AI safety specifically?
Yes. The format is ideal for AI safety because it lets you test responses to issues like bias, data leakage, or unintended behavior without real-world harm.
Related
Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.