Monitor AI Agents for Failures, Unsafe Actions & Costs
This guide shows you how to set up practical monitoring for AI agents—catching failures, unsafe actions, and runaway costs before they become big problems. You’ll learn what to track, which alerts to configure, and how to respond when something goes wrong. No prior monitoring experience is needed, but you should have access to your agent’s logs and any cloud console where it runs.
What You’ll Need
- Access to your AI agent’s logs (e.g., stdout, API call logs, or a logging service like CloudWatch, Datadog, or a local file).
- A way to set up alerts (e.g., email, Slack, PagerDuty, or your cloud provider’s alerting tool).
- If your agent uses external APIs (like LLM calls), access to your API usage dashboard or billing page.
- Optional: a simple dashboard tool (Grafana, Excel, or even a spreadsheet) to visualize trends.
Step-by-Step
- Define what “failure” and “unsafe” mean for your agent.
Write down concrete conditions. For example: a step that throws an error, a response that contains a banned word, an action that tries to delete a file, or a request that exceeds a time limit. This becomes your checklist for monitoring. - Enable detailed logging.
Make sure your agent logs every input, output, action taken, and the reasoning behind it (if available). Include timestamps, user IDs (if any), and a unique run ID. If you’re using a framework like LangChain or AutoGen, turn on verbose logging. - Log token usage and API costs.
For every LLM call, record the model, token count (input/output), and cost. Most SDKs return this info; if not, estimate using the model’s per-token price. Store this in a structured format (e.g., JSON lines) for later analysis. - Set up real-time alerting for errors.
Create alerts that trigger when: an exception is thrown, a step retries more than 3 times, or a tool returns an error code. Use your logging service’s alert rules—e.g., “count of ERROR level logs > 5 in 5 minutes.” - Monitor for unsafe actions with a rule-based filter.
Implement a simple keyword/pattern filter on the agent’s actions. Flag actions like: file deletion, sending emails, making purchases, or accessing sensitive data. Log these as “WARN” and alert if they happen without approval. - Track cost in real time.
Set a budget per run and per day. Use a counter that accumulates token costs. If the cost exceeds a threshold (e.g., $10 per run), pause the agent and alert. You can do this in code or via your cloud billing alerts. - Create a dashboard for trends.
Visualize: success rate, average latency, cost per run, and number of unsafe actions over time. This helps you spot gradual degradation before it becomes critical. - Test your monitoring.
Run a few test scenarios: force an error, trigger a keyword, and simulate a cost spike. Make sure alerts fire and you receive them. Adjust thresholds if you get too many false positives. - Document a response plan.
Write down what to do when each alert triggers: who to contact, whether to pause the agent, and how to roll back a change. Keep this in a runbook. - Review and tune regularly.
Once a week, look at the alerts that fired and the dashboard. Adjust thresholds and filters to reduce noise and catch new failure modes.
Common Mistakes
- Only monitoring errors, not unsafe actions. Many agents fail silently—they complete a task but in a harmful way. Always log actions, not just exceptions.
- Ignoring token usage until the bill arrives. Cost spikes can happen in minutes. Set real-time alerts.
- Too many alerts. If every tiny error triggers a page, you’ll ignore them. Use severity levels and aggregate alerts.
- No baseline. Without historical data, you won’t know what “normal” looks like. Start logging early.
- Forgetting to monitor the monitoring. If your log ingestion fails, you’re blind. Add a heartbeat check.
Tips
- Use structured logging (JSON) so you can query easily.
- Add a unique run ID to every log line—this lets you trace a single agent’s entire journey.
- For unsafe actions, use a “human-in-the-loop” approval step. The agent pauses and asks for confirmation before executing a risky action.
- Set a hard cap on the number of steps an agent can take per run to prevent infinite loops.
- Use separate alert channels: critical (page) vs. warning (email/Slack).
- If you use a cloud provider, take advantage of their budget alerts (e.g., AWS Budgets, Azure Cost Alerts).
Troubleshooting
- Alerts not firing? Check that your log level is set correctly (e.g., INFO vs. ERROR). Also verify the alert query matches the log format. Test with a manual log entry.
- Too many false positives? Increase the threshold or add more specific filters. For example, instead of flagging all “delete” actions, only flag deletions of files outside a specific directory.
- Cost tracking is inaccurate? Ensure you’re capturing all API calls—sometimes retries or tool calls are missed. Cross-check with your provider’s billing dashboard.
- Logs are too noisy? Use log sampling for high-volume, low-value logs, but keep full logs for error and warning levels.
- Agent is already in a runaway loop? Implement a “circuit breaker” that stops the agent after a max number of steps or a max cost. You can add this in code even after deployment.
- Dashboard shows no data? Check that your log stream is connected and that the time range is correct. Also verify the metric names match.
Frequently Asked Questions
How do I monitor an AI agent that doesn’t have built-in logging?
Wrap the agent’s function calls with your own logging. For example, if it’s a Python script, add a decorator that logs the input, output, and duration of each call. For API-based agents, use a proxy that logs all requests.
What’s the best way to detect unsafe actions?
Start with a simple rule-based system: a list of forbidden patterns (e.g., “rm -rf”, “DELETE FROM”). For more advanced detection, use a separate AI model to review the agent’s actions, but that adds cost.
How do I set a budget limit for an AI agent?
In your code, before each API call, check the cumulative cost. If it exceeds the limit, stop the agent and log the reason. You can also use your cloud provider’s budget alerts, but those have a delay.
What metrics should I track for cost?
Track: cost per run, cost per step, token count per call, and cumulative daily cost. Also track the number of API calls to spot unusual spikes.
How often should I review the monitoring setup?
At least weekly. As your agent’s behavior changes (e.g., new tools, new prompts), your monitoring thresholds may need adjustment.
Can I monitor an agent that runs on a local machine?
Yes, use local logging (e.g., log files) and a tool like Prometheus + Grafana, or even a simple script that sends alerts via email. For cost, you’ll need to manually track API usage from your provider’s dashboard.
Related
Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.