How to Eval a Tool-Calling Agent's Correctness
Tool-calling agents fail in ways a final-answer check can't see. An agent can return a fluent, correct-sounding response while having called the wrong tool, passed a subtly wrong argument, or guessed a value it should have asked about. Because the run completes without an error, these failures look identical to success until someone inspects the actual call. Evaluating tool-calling correctness means scoring the call the agent made — the tool it chose and the arguments it passed — not just the sentence it produced afterward.
Quick answer: Test a tool-calling agent on four things, scored independently: did it pick the right tool, did it pass the right arguments (compared after normalizing surface form, not by brittle exact-match), did it ask for clarification instead of guessing on an ambiguous request, and did it avoid calling a tool when none was needed. Score the call the agent makes, not the answer it writes — a wrong tool call frequently still produces a plausible-looking final response.
What should a tool-calling eval actually measure?
Score these as separate checks. Collapsing them into one pass/fail number hides which capability is actually broken.
| Check | Question it answers | Failure it catches |
|---|---|---|
| Tool selection | Did the agent choose the right tool for the request? | Calling search when the task needed book_appointment |
| Argument accuracy | Are the argument values correct, not just well-typed? | Right tool, wrong date; a valid-but-hallucinated ID |
| Clarification / abstention | Did it ask instead of guess on an ambiguous request? | Confidently inventing a missing parameter |
| No-op discipline | Did it avoid calling a tool when none was needed? | Reflexively invoking a tool on a plain question |
A schema-valid call that names the wrong tool passes a naive function-calling check and fails the task. That gap — syntactically perfect, semantically wrong — is where most real tool-calling failures live.
How do you test whether an agent asks instead of guessing?
This is the tool-calling analog of abstention in a RAG agent, and it's the check homegrown evals most often skip. Include deliberately under-specified requests in the test set — a booking request with no date, a lookup with an ambiguous name — where the correct behavior is to ask a clarifying question, not to call a tool with an invented argument.
An agent that fills the gap with a confident guess is the costly failure mode: it will happily book the wrong slot or email the wrong person, and the run looks successful. An eval that only ever asks fully-specified questions never exercises this behavior, so it can't tell a careful agent from a reckless one.
How do you know your tool-calling eval actually works?
Run the candidate eval against agents whose quality you already know, before you trust its verdicts on an agent you don't. This is the heart of a credible eval — mutation testing applied to the eval itself:
- A known-good reference agent (right tool, right arguments, asks when unsure) should score high.
- A known-broken reference agent (loose argument discipline, occasionally guesses) should score visibly lower.
- A known-sabotaged reference agent (deliberately calls plausible-but-wrong tools) should score near the floor.
If the eval can't separate those three, it isn't measuring tool-calling correctness — it's measuring something merely correlated with it. See the tool-calling correctness benchmark for this discriminating-power check run on a real reference panel, and what a reference-panel harness is for the method in depth.
Does exact-match on tool arguments work?
Rarely, on its own. Exact string comparison fails an agent for writing "New York City" instead of "NYC", or an ISO date instead of a spelled-out one — both correct values, different surface forms. That inflates the failure rate with false negatives and, worse, makes the eval non-deterministic in spirit: a harmless phrasing change flips the verdict.
A robust argument check normalizes first (dates to a canonical form, known aliases to a canonical value) and compares semantically where it can, so it fails only genuine value errors. It should also stay deterministic: the same call scored twice must produce the same verdict, or regressions become invisible in CI. Determinism and ungameability are structural checks the eval must pass before its scores mean anything — see how to detect a gameable eval.
Related
- What is a reference-panel harness — the discriminating-power method the "does the eval work" check is built on.
- How to detect a gameable eval — the determinism and ungameability checks referenced above.
- How to eval a RAG agent's groundedness — the same rubric approach applied to a different capability.
- Capability packs — the category tool-calling correctness belongs to.
- Tool-calling correctness packs — browse the method.
- The tool-calling correctness pack — a verified pack implementing this exact rubric.
- Tool-calling correctness benchmark — the discriminating-power results referenced above.
FAQ
Function-calling accuracy usually means the model emitted syntactically valid JSON that matches the schema. Correctness is broader and harder: it also asks whether that was the right tool for the request, whether the argument values are right (not just well-typed), and whether the agent should have asked a question instead of calling anything at all.
Only when the request was unambiguous. On a genuinely under-specified request, asking is the correct behavior and should score full marks — an agent that confidently guesses arguments on an ambiguous request is exhibiting the exact failure the eval should catch, not rewarding.
Yes, and usually you should. Give the agent tool schemas and score the call it chooses — tool name plus arguments — against the expected call. This keeps the eval deterministic and free of side effects. Execute real or simulated tools only when you also need to test how the agent handles the returned result.
It scales with the risk of the deployment, not a fixed number. A low-risk internal helper can be meaningful at ~10 cases; an agent that can move money or change production state needs many more. AgentGrading scores test-case volume against a minimum that rises with the pack's own risk tier, so a high-risk pack with only 15 cases scores worse than a low-risk one with 15.
Often, yes. "NYC" and "New York City", or 2026-07-06 and July 6 2026, are the same argument in different surface forms. A good eval normalizes before comparing, or checks semantic equivalence, so it fails an agent for choosing the wrong value — not for choosing a different valid spelling of the right one.
Silent wrong-tool calls that still return plausible output. If an agent calls a search tool when it should have called a booking tool, the run often completes without an error — the final answer just quietly does the wrong thing. Scoring only the final response, instead of the actual call, misses this entirely.