How to Audit an AI Agent’s Changes to Your Server
This guide explains how to systematically review and verify the changes an AI agent makes to your server, ensuring nothing unintended or harmful slips through. You’ll learn a practical, step-by-step audit process that covers everything from initial setup to rollback, plus how to spot common mistakes and troubleshoot issues. Before you start, you’ll need root or sudo access to the server, a basic understanding of the command line, and a clear record of what the agent was asked to do.
What You’ll Need
- Root or sudo access to the server you want to audit.
- SSH client (like Terminal on macOS/Linux or PuTTY on Windows).
- A log of the AI agent’s sessions or commands (if available).
- A backup or snapshot of the server taken before the agent started (highly recommended).
- Basic familiarity with the command line and common server tools.
Step-by-Step
- Define the scope of the audit. Write down exactly what tasks the AI agent was supposed to perform. For example, “deploy the new version of the app” or “add a user account for John.” This gives you a baseline to compare against.
- Take a snapshot or backup (if you haven’t already). If the agent is still running, stop it. Then create a fresh backup or snapshot of your server. This is your safety net—if anything goes wrong during the audit, you can restore from this point.
- Review the agent’s logs. Look for any logs the AI agent might have produced (e.g., in
/var/log/agent.logor in the agent’s dashboard). Note the timestamps of its actions. If you don’t have logs, skip to the next step. - Check system logs for the relevant time window. Use
journalctl --since "2025-01-15 10:00" --until "2025-01-15 11:00"(adjust times) to see what happened on the system while the agent was active. Look for unusual entries like new user creation, service restarts, or package installs. - List all recent file changes. Run
find / -xdev -mtime -1 -type f 2>/dev/null | grep -v '/proc\|/sys\|/dev'to find files modified in the last 24 hours. Review the list for anything unexpected—new files in/etc,/usr/local/bin, or/var/wwware especially important. - Check user accounts and groups. Run
cat /etc/passwdandcat /etc/group. Look for new users or changes to existing ones. A new user with a UID of 0 (root) is a red flag. - Review scheduled tasks (cron jobs). Check
/etc/crontab,/etc/cron.d/, and each user’s crontab (crontab -l). Look for new entries that could be malicious or unintended. - Inspect running services and processes. Run
systemctl list-units --type=service --state=runningandps aux. Compare against a known baseline if you have one. Look for new services or processes that seem out of place. - Check network listeners. Run
ss -tulpnto see open ports. New listeners could indicate backdoors or unintended services. - Review package installations. On Debian/Ubuntu, check
grep ' install ' /var/log/dpkg.logfor the time period. On RHEL/CentOS, useyum historyordnf history. Remove any packages that weren’t part of the task. - Compare configuration files. If you have backups, use
diffto compare key files like/etc/ssh/sshd_config,/etc/nginx/nginx.conf, or/etc/my.cnfagainst the originals. Look for changes in permissions, authentication methods, or other security settings. - Check for new SSH keys. Look in
/root/.ssh/authorized_keysand all user home directories. Any new keys should be verified with the user who owns them. - Verify database changes (if applicable). If the agent had database access, check for new tables, rows, or changes to stored procedures. Use the database’s query logs if available.
- Review any scripts or code the agent created. If the agent wrote files, read them. Look for suspicious patterns like base64-encoded strings, curl/wget calls to unknown URLs, or attempts to disable security tools.
- Document your findings. Create a report listing every change you found, whether it was expected, and any action taken. This is your audit trail.
- Roll back any unintended changes. If you find something that shouldn’t be there, revert it. Use your backup or manually reverse the change. For example, if a new user was created, remove them with
userdel. - Test the server. After your audit, verify that the server still functions as expected. Run your application’s health checks, try logging in, and ensure services are running.
Common Mistakes
- Not taking a backup before the audit. If you make a mistake during rollback, you’ll have no way to recover.
- Only checking file changes, not user accounts or services. AI agents can make changes in many places; a thorough audit covers all areas.
- Ignoring the agent’s logs. The agent’s own logs often show exactly what it did, making the audit much easier.
- Assuming the agent only did what it was asked. Always verify, even if you trust the tool.
- Rolling back changes without understanding what they were for. You might break something the agent correctly set up.
Tips
- Use a version control system for your configuration files (e.g., Git) to make tracking changes easier.
- Run the agent in a staging environment first, if possible, to see what it does before letting it touch production.
- Set up file integrity monitoring (like AIDE or Tripwire) to catch changes automatically.
- Keep a baseline of your server’s state (e.g., list of packages, running services) to compare against.
- If the agent supports it, enable a “dry run” mode to preview changes before they’re applied.
Troubleshooting
I can’t find any logs from the agent. If the agent doesn’t produce logs, rely on system logs and file timestamps. You can also check shell history (~/.bash_history) if the agent ran commands as a user.
The agent’s changes broke the server. Restore from your backup immediately. If you don’t have a backup, try to reverse the most recent changes you identified. For example, if you see a new package was installed, remove it. If a service was stopped, start it again.
I see a change but I’m not sure if it was the agent. Check the timestamp against the agent’s activity window. If it falls within that window, it’s likely the agent. If not, investigate further—it could be a separate issue.
The audit is taking too long. Prioritize the most critical areas: user accounts, SSH keys, cron jobs, and running services. These are the most common targets for malicious changes.
I can’t tell if a file change is legitimate. Compare the file against a known good copy (from your backup or package manager). If it’s a configuration file, look for comments explaining the change. If in doubt, revert and see if anything breaks.
Frequently Asked Questions
How often should I audit AI agent changes?
Ideally after every significant task the agent performs. At minimum, do a weekly spot-check if the agent runs regularly.
Can I automate the audit process?
Yes, you can write scripts that check for common changes (e.g., new users, new cron jobs) and alert you. But automated checks can’t catch everything, so manual review is still important.
What if the agent has root access? Should I still audit?
Yes, especially if it has root access. That’s when auditing is most critical, because the agent can change anything.
How do I know if a change is malicious or just a mistake?
Look at the intent: is the change consistent with the task you gave? If you asked it to deploy a web app and it created a new user, that’s suspicious. If it created a new directory in /var/www, that’s probably fine.
What should I do if I find a malicious change?
Immediately revert the change, then change all passwords and SSH keys on the server. Consider restoring from a backup taken before the agent started. Also, stop using the agent until you understand how it was compromised.
Is it safe to let an AI agent have sudo access?
It’s risky, but sometimes necessary. If you do, make sure you have strong logging and auditing in place, and consider restricting the agent to specific commands using sudoers rules.
Related
Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.