Coding Agent Model Harness and Tool Loop Explained
A coding agent combines a language model with a harness: the software that supplies context, runs tools, and manages the conversation. Understanding the model-and-tool loop helps you build agents that can inspect a codebase, make changes, run checks, and report what happened without treating model output as automatically trustworthy.
What You’ll Need
- A model endpoint or local model that supports tool use, or can produce structured tool-call requests.
- A harness application that can send prompts, interpret model responses, execute approved tools, and return results to the model.
- A working directory or repository and a small set of tools, such as reading files, searching text, editing files, and running tests.
- Clear permission rules. Start with a disposable project or read-only access, and require approval before consequential actions.
Step-by-Step
- Separate the model from the harness. The model generates a response based on its input; it does not inherently see your filesystem or execute commands. The harness is the surrounding program. It builds the model request, exposes tool descriptions, validates proposed calls, runs permitted tools, and sends results back. Keep that boundary explicit in your design.
- Define the task and its limits. Give the agent a concrete goal, relevant repository context, and constraints such as which files it may change, whether it may access the network, and which commands need approval. A focused task like “find why this test fails and propose a minimal fix” is easier to supervise than “improve the project.”
- Assemble the model request. The harness usually combines instructions, the user’s task, conversation history, and available tool definitions into a request. Tool definitions describe a name, purpose, and input schema—for example, a file-reading tool that requires a path. Include only useful context: large histories and excessive file contents consume the model’s context window and can obscure important details.
- Ask the model for its next response. The model may return ordinary text, a structured request to call a tool, or a final response. Treat the response according to the API’s documented format rather than guessing from prose. A sentence saying “I ran the tests” is not proof that a test tool actually ran.
- Validate each proposed tool call. Check that the tool exists, arguments match its schema, paths stay within allowed boundaries, and the requested action fits the permissions for this task. Reject malformed or disallowed calls, and return a clear error to the model when it can safely recover. Validation belongs in the harness, not just in the model instructions.
- Execute the tool in a controlled environment. Run the approved action and capture its result, such as file contents, search matches, exit status, or test output. Apply timeouts and output limits; avoid exposing secrets in logs or tool results. For shell access, use the least privilege practical and consider a sandbox, especially for untrusted repositories or commands.
- Return the result as a tool response. Add the tool’s output to the conversation in the format the model interface expects, linked to the corresponding call when required. Distinguish actual output from harness-generated messages. If a command fails, return its exit status and relevant error text instead of silently presenting it as successful.
- Continue the loop until a stopping condition. Send the updated conversation back to the model. It may inspect another file, edit code, run a test, or answer the user. Repeat validation and execution for every call. Stop when the model provides a final answer, the task reaches a limit, the user cancels, or a safety or runtime condition requires stopping.
- Review changes and report evidence. Inspect the final diff and run appropriate checks before accepting changes. Tell the user what was changed, which checks actually completed, and what remains uncertain. If the agent could not run a test, say so rather than implying that the code is verified.
Common Mistakes
- Assuming tool access is automatic: A model only interacts with tools that the harness exposes and executes.
- Trusting generated arguments: Model-produced paths and commands need validation, permission checks, and appropriate containment.
- Confusing a tool request with a result: A call is a proposal. Only the harness can execute it and observe its actual outcome.
- Returning vague tool errors: “Something failed” gives the model little to work with. Include concise, factual error details and status information.
- Letting the loop run indefinitely: Set limits for turns, tool calls, execution time, and output size, with a clear stop path.
- Giving broad access by default: Unrestricted shell, network, or write access increases the impact of mistakes. Grant only what the task needs.
Tips
- Use narrow tools with typed inputs instead of one unrestricted command tool wherever practical.
- Keep a record of tool name, validated arguments, result status, and timing. Redact credentials and sensitive data.
- Make file edits reviewable: use diffs, preserve a clean working tree when possible, and avoid overwriting unrelated changes.
- Use explicit approval gates for deletion, external writes, package installation, network access, or other consequential operations.
- Design tool output for the next decision. Return relevant excerpts and errors rather than dumping entire large files or logs.
- Test failure paths too: malformed arguments, denied permissions, timeouts, empty results, and interrupted model requests.
Troubleshooting
The model never calls a tool. Check that tool definitions are included in the request, their schemas are valid, and the model interface is configured to allow tool calls. Make the task clearly require an action the tool can perform.
A tool call is rejected repeatedly. Compare the proposed name and arguments with the actual schema. Check required fields, types, allowed paths, and permission rules. Return a precise explanation of the rejection, but do not weaken a safety rule just to complete the call.
The tool runs but the model acts as if it did not. Verify that the harness returns the result with the correct call reference and role or message type expected by the model API. Include useful output and a clear failure status.
The agent repeats calls or does not finish. Inspect the conversation history and stop conditions. Confirm that results are not being dropped and that the model can see prior tool outcomes. Add a maximum turn or call count and return a concise status when it is reached.
Tests fail or changes look wrong. Separate agent reasoning from tool evidence: inspect the diff, reproduce the command in the intended environment, and share the exact failure output with the model if another attempt is appropriate. Keep a way to revert changes.
Frequently Asked Questions
Does the model execute commands itself?
No. The model can request a tool call; the harness decides whether to run it and returns the result.
What is a tool loop?
It is the repeated cycle of sending context to the model, validating and executing a requested tool, returning its result, and asking the model what to do next.
Can an agent work without tools?
Yes, it can answer from supplied context, but it cannot independently inspect files, change code, or run tests without tools provided by a harness.
Should every tool call require human approval?
Not necessarily. Low-risk, read-only actions can often be allowed within limits; consequential or broad actions should have stronger controls or explicit approval.
How does the harness know when to stop?
It can stop when the model returns a final answer, the user cancels, or a configured limit or safety condition is reached. Always define these conditions.
Is a successful test run proof that the code is correct?
No. It is evidence that the selected tests passed in that environment. Tests may be incomplete, so report what ran and retain human review where needed.
Related
Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.