ThisCalculator

Everyday

Audit & Secure an Unattended Coding Agent

Updated · Standard formula, shown below

This guide walks you through auditing and securing an unattended coding agent that has write access to your repository. You’ll learn how to review what the agent has done, lock down its permissions, and set up guardrails to prevent accidental or malicious damage. Start by ensuring you have admin access to the repository and the agent’s configuration, and a way to review its commit history.

What You’ll Need

  • Admin or owner access to the repository hosting the agent.
  • Access to the agent’s configuration files (e.g., CI/CD pipeline, cron job, or server setup).
  • A way to view commit history and logs (e.g., GitHub, GitLab, Bitbucket, or local git).
  • Optional: a separate test repository to practice changes.

Step-by-Step

  1. Inventory all agent access points. List every place the agent can interact with the repository: direct git pushes, pull request creation, branch merges, issue comments, or API calls. Check for any tokens, SSH keys, or service accounts the agent uses.
  2. Review recent activity. Examine the agent’s commits, PRs, and branches. Look for unexpected changes, large diffs, or modifications to sensitive files like package.json, requirements.txt, or .env. Use git log --author="agent-name" to filter its activity.
  3. Audit permissions. For each access point, determine the minimum permissions needed. Remove write access to protected branches (like main or prod) if not required. Use repository settings to restrict the agent to specific branches or paths.
  4. Rotate credentials. If you find any tokens or keys, revoke them and issue new ones with tighter scopes. For GitHub, create a fine-grained personal access token with only the necessary repository permissions. For SSH keys, generate a new key pair and update the agent’s configuration.
  5. Set up branch protection. Enable branch protection rules on critical branches. Require pull request reviews, status checks, and prevent direct pushes. This forces the agent to create PRs instead of committing directly, giving you a review point.
  6. Implement code review for agent changes. Configure the agent to open a PR for every change, even if it’s a small fix. Add a human reviewer or an automated CI check that runs tests and linters before merging.
  7. Add logging and alerting. Enable audit logs for the repository (e.g., GitHub Audit Log) and set up alerts for unusual activity, such as force-pushes or changes to protected files. Use webhooks or CI triggers to notify you on every agent action.
  8. Test the agent in a sandbox. Before allowing the agent to write to the real repository, run it against a fork or a test branch. Verify it behaves as expected and doesn’t attempt to access secrets or modify unrelated files.
  9. Document the agent’s purpose and allowed actions. Write a clear policy stating what the agent is allowed to do (e.g., update dependencies, fix lint errors) and what it must never do (e.g., modify CI config, push to main). Share this with your team.
  10. Schedule regular audits. Set a recurring calendar reminder (e.g., monthly) to repeat steps 1-3. As the agent’s code evolves, its permissions may need adjustment.

Common Mistakes

  • Giving the agent admin permissions just to simplify setup. Always use the least privilege.
  • Ignoring the agent’s dependency updates. An agent that updates packages can introduce vulnerabilities or break builds. Always review dependency changes carefully.
  • Not rotating credentials after a breach or employee departure. If a token was exposed, assume it’s compromised.
  • Allowing the agent to push directly to production branches. This bypasses all review and can cause downtime.
  • Forgetting to monitor the agent’s activity. Without logs, you won’t know if it’s misbehaving until it’s too late.

Tips

  • Use a dedicated service account for the agent, not a personal account, so you can track its actions separately.
  • If your platform supports it, use a “require approval” setting for agent-initiated PRs.
  • Keep the agent’s dependencies and runtime updated to avoid security holes.
  • Consider using a read-only token for the agent’s initial setup, then switch to write permissions only when needed.
  • Use a separate branch naming convention (e.g., agent/<task>) to easily identify agent work.

Troubleshooting

  • The agent can’t push after tightening permissions. Check the token scope and branch protection rules. Ensure the agent’s user is allowed to push to the target branch (if not protected) or that PRs are allowed.
  • The agent is making unwanted changes. Review its configuration and instructions. It may be following a vague prompt; clarify its scope in the agent’s setup.
  • You see unexpected commits from the agent. Check if the agent’s credentials were used by someone else. Rotate them immediately and review the commit diffs for malicious content.
  • CI checks fail on agent PRs. The agent may be generating code that doesn’t pass linting or tests. Adjust the agent’s instructions or fix the CI configuration to allow known-good patterns.
  • You can’t find the agent’s activity in logs. Ensure audit logging is enabled and that you’re looking at the right repository. Some platforms log under the service account name.

Frequently Asked Questions

Can I let an unattended agent have write access to my repository at all?

Yes, but it’s risky. With proper permissions, branch protection, and monitoring, you can reduce the risk. Always use the principle of least privilege and require human review for critical changes.

What’s the safest way to give an agent write access?

Use a fine-grained token with write access only to specific branches or paths, enable branch protection, and require PRs for all changes. Avoid giving admin or owner permissions.

How do I know if the agent has been compromised?

Watch for unusual commit times, unexpected file changes, or activity outside its normal scope. Check audit logs regularly and set up alerts for suspicious actions like force-pushes or secret access.

Should I use a separate GitHub account for the agent?

Yes. A dedicated service account makes it easy to track the agent’s actions and revoke access without affecting personal accounts.

What should I do if the agent accidentally deletes a branch?

If you have branch protection, this shouldn’t happen. If it does, use the repository’s reflog or a backup to restore the branch. Then tighten permissions to prevent recurrence.

Can I limit the agent to only certain files or directories?

Yes, some platforms support path-based permissions. For example, in GitHub you can use CODEOWNERS or branch rules to restrict changes to specific paths. Otherwise, you can configure the agent’s instructions to avoid touching certain files.

Related

Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.