Monitoring AI Agents for Unsafe Actions
This guide explains how to set up monitoring for AI agents to catch unsafe or unintended actions before they cause harm. You’ll learn practical steps to define what to watch, collect the right logs, set up alerts, and respond when something goes wrong. No special hardware is needed—just access to your agent’s logs and a place to store and review them.
What You’ll Need
- Access to your AI agent’s logs (action history, inputs, outputs, and system events)
- A logging or monitoring system (even a simple spreadsheet or text file works for small setups)
- A way to set up alerts (email, Slack, or a simple script)
- Time to review logs regularly
Step-by-Step
- Define what “unsafe” means for your agent. Write down specific actions that are unacceptable. Examples: deleting files, sending emails without approval, making purchases, accessing sensitive data, or taking irreversible system actions. Be as concrete as possible—vague rules like “be safe” won’t help you detect issues.
- Enable comprehensive logging. Turn on logging for every action the agent takes. Include timestamps, the exact action (e.g., API call, file operation), the inputs used, the outputs returned, and any error messages. If your agent framework has built-in logging, enable it. If not, add logging wrappers around the agent’s execution loop.
- Log the agent’s reasoning (if available). Many agents can output their chain-of-thought or decision rationale. Log this too—it helps you understand why an action was taken. If your agent doesn’t expose reasoning, log the context it received (e.g., system prompt, user input, tool results).
- Set up structured log storage. Store logs in a searchable format. A simple CSV or JSON file works for small volumes. For larger deployments, use a log management tool like ELK, Splunk, or even a SQL database. The key is that you can search and filter logs quickly.
- Create a list of “red flag” patterns. Write regex patterns or keywords that indicate unsafe actions. Examples: “rm -rf”, “DELETE FROM”, “send_mail”, “transfer_funds”, “ssh”, “chmod 777”. Also include patterns for actions that require human approval, like “purchase” or “create_user”.
- Implement automated alerts. Write a script that scans new logs for red flags and sends an alert (email, Slack message, or SMS) when a match is found. Include the log entry and a link to the full context. Test the alert system with a fake log entry to ensure it works.
- Set up a human review process. Designate someone (or a team) to review alerts and logs regularly. For low-volume systems, a daily review is fine. For high-risk agents, review in real-time. Create a checklist for what to look for: unexpected actions, actions outside the agent’s scope, unusual timing, or repeated failures.
- Add a “dry-run” or sandbox mode. If possible, run the agent in a sandboxed environment that mimics production but blocks irreversible actions. Monitor the dry-run logs to see what the agent would do. This is especially useful before deploying a new agent or updating its prompts.
- Implement a kill switch. Have a way to immediately stop the agent if something goes wrong. This could be a script that terminates the agent’s process, a flag that pauses execution, or a remote shutdown API. Test the kill switch regularly.
- Review and update your monitoring rules regularly. As your agent evolves, new risks may emerge. Schedule a monthly review of your red flag patterns and alert thresholds. Update them based on incidents you’ve seen and new capabilities you’ve added.
Common Mistakes
- Only monitoring errors, not actions. Many people log only when the agent fails. But unsafe actions often succeed. Log every action, not just failures.
- Alert fatigue. If you set too many alerts, you’ll start ignoring them. Focus on high-risk actions first, and tune the thresholds.
- Not logging inputs and outputs. Without context, you can’t tell if an action was justified. Always log the full input and output of each action.
- Forgetting to monitor the monitoring. If your logging system fails, you’re blind. Check that logs are being written and alerts are firing.
- Ignoring near-misses. If the agent almost did something unsafe but was blocked, that’s a warning sign. Review near-misses to find weaknesses in your guardrails.
Tips
- Start with a simple monitoring setup and expand as you learn what matters.
- Use a separate log file for safety-critical actions to make review faster.
- Include a unique request ID in every log entry to trace a single user request across all actions.
- Automate the review of common safe actions (e.g., “read file”) so humans only see unusual ones.
- Document your monitoring rules so that new team members can understand them.
Troubleshooting
- No logs are being generated. Check that the agent’s logging is enabled and that the log file path is writable. Test with a simple action.
- Alerts are not firing. Verify that your script is running (check cron jobs or scheduled tasks), that the regex patterns match the log format, and that the alerting service (e.g., email) is configured correctly.
- Logs are too large. Implement log rotation (e.g., keep last 7 days) and archive older logs to a cheaper storage. Use sampling for high-volume, low-risk actions.
- False positives. If you get too many alerts for safe actions, refine your patterns. For example, exclude actions that only read data or that occur in a sandbox.
- Agent bypasses monitoring. If the agent can execute code that disables logging, you need to run the agent in a restricted environment where it cannot modify its own logging. Use separate user accounts and file permissions.
Frequently Asked Questions
What is the most important thing to monitor?
The most important thing is the agent’s actions that have side effects—anything that changes state, sends data, or affects the real world. Reading data is lower risk, but still log it for context.
Can I monitor AI agents without technical skills?
Yes, for simple agents you can manually review logs in a text file. For automation, you’ll need some basic scripting skills, but you can start with a simple email alert using a service like IFTTT.
How often should I review logs?
It depends on the risk. For a chatbot that just answers questions, a weekly review is fine. For an agent that can execute code or send emails, review daily or even in real-time.
What if the agent takes an unsafe action before I catch it?
Have a rollback plan. For example, take regular backups of any data the agent can modify, and have a way to undo actions like sending emails (e.g., use a delay queue).
Should I monitor the agent’s internal reasoning?
Yes, if available. It helps you understand why an action was taken, which is crucial for fixing the underlying cause. If not available, log the context (prompt, tool results).
Can I use the same monitoring for all types of agents?
The principles are the same, but the specific red flags will differ. A code-writing agent needs different patterns than a customer service agent. Tailor your rules to each agent’s capabilities.
Related
Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.