It's not your token bill
For most people, an MCP server's token cost will never show up on an invoice. Clients cache tool definitions, and more of them now load definitions only when a task needs them. On Anthropic's API, cached reads are heavily discounted and don't count against input rate limits. So if the case for efficient MCP tools were money alone, it would be a weak case.
It isn't. The real cost of a clumsy MCP server is paid in accuracy, failed runs, waiting time and the usage limit on the plan you already pay for. An agent that needs 30 tool calls where 10 would do has more chances to pick the wrong tool, guess a parameter, or stop halfway through a change. Here is what the evidence says, what it doesn't say yet, and how we measure it.
1. Big tool catalogs make the right tool harder to find
An agent's context is shared between your code, your instructions and every tool definition the client exposes. Lazy loading (Claude Code, for example, can defer MCP tool definitions and search for them on demand) cuts the repeated cost. It doesn't remove the harder problem: the agent still has to find and choose the right tool among everything installed.
- Chroma tested 18 models and found that reliability drops as input grows, even on simple tasks, and that a single distracting passage is enough to hurt accuracy. That's a study of long context in general, not of tool catalogs, but tool definitions are part of that context.
- Anthropic measured five common MCP servers at 58 tools and about 55K tokens of definitions before the conversation starts. In its internal MCP evaluations, loading tools on demand instead of all at once raised Opus 4 from 49% to 74%. That's one vendor evaluating its own tool-loading feature, not an independent test.
- MCP-Universe connected 7 servers (94 tools) to every task instead of only the servers each task needs. Claude 4 Sonnet's success on location navigation fell from 22.22% to 11.11% (10 to 5 of 45 tasks), GPT-4.1's on browser automation from 23.08% to 15.38% (9 to 6 of 39), and GPT-4.1's on financial analysis from 40% to 35% (16 to 14 of 40). Each comparison rests on 39 to 45 tasks, and all three point the same way.
- How many tools should an MCP server have? There's no universal number. OpenAI suggests aiming for fewer than 20 functions available at the start of a turn, as a soft guideline; Anthropic recommends "a minimal viable set of tools". What matters is how many definitions the client exposes at once, whether each one does a distinct job, and whether the agent can find the right one when it needs it.
For a team: your agent's attention is split between your codebase and every server you've installed. For an individual: every token a tool doesn't take is a token your code and instructions get.
2. Wrong tool, wrong parameters, and server design fixes both
The most common way an agent fails with tools isn't a hard reasoning problem. It picks the wrong tool or guesses an argument.
- Anthropic names wrong tool selection and incorrect parameters as the most common failures, "especially when tools have similar names". Adding examples to tool definitions took parameter accuracy from 72% to 90% in its testing.
- On LiveMCP-101, semantic parameter errors run 16–25% even for strong models.
- A study of 856 tools across 103 MCP servers found that 97.1% of tool descriptions had at least one "smell" (a part of the description scoring below minimum-viable on the authors' rubric), with no statistically significant difference between official and community servers. On the 202 tools of the MCP-Universe benchmark, improving the descriptions raised median task success by 5.85 points, but also added 67.46% more execution steps and made results worse in 16.67% of cases. Better descriptions help, and they aren't free.
We saw this in our own audit of GitHub's MCP server. To label or assign an issue, agents first reached for two tools that couldn't do it, then found the one that could: ui_get, described as "Fetch UI data for MCP Apps". A description that says what the tool is for ("List the labels, assignees, milestones and reviewers that can be set on an issue or pull request") would have sent them there first. That's a design bug in the server, not in the prompt.
3. Every avoidable turn is a chance to fail, and failures are the expensive path
- In GitHub Copilot's agent telemetry, 9% of agent turns included a tool failure, and retry loops multiplied compute by up to 4×. Those are coding-agent tool failures in general, not MCP failures specifically.
- SWE-Effi: on SWE-bench, unresolved attempts tended to use more resources than resolved ones; for SWE-Agent with GPT-4o-mini, more than 4× as much. The authors name "the number of interactions, not the intensity of each one" as the main cost driver.
- Vercel cut an internal agent from about 17 tools to 2: steps fell from about 12 to 7, time from 275 s to 77 s, and success rose from 4/5 to 5/5. It changed the agent's architecture at the same time, and it's five queries, so read it as a case study, not a controlled test of tool count.
More calls don't always mean lower success. In Speakeasy's own benchmark of its dynamic toolsets, the design used 2–3× more calls and ran about 50% slower while keeping its tool-selection success rate. What matters is avoidable turns: paging back and forth, fetching a list and then calling once per item, or re-finding an ID the server already had. In our GitHub audit, a pull-request review with line comments took 5 calls through the MCP server; GitHub's REST API accepts the same review in one.
4. Most of the waiting is model time
In Copilot's telemetry, model inference took about 85% of wall-clock time and tool execution under 5%. That doesn't mean every call removed saves the same amount of time: tools can run in parallel and overlap with the model. It does mean a tool's own speed is rarely the bottleneck. The number of sequential model cycles usually is. A four-minute run is a context switch; a one-minute run keeps the engineer in the loop.
5. Your plan's limit is measured in tokens, and clients now show who used it
This is the reason that matters most to individuals. Claude Code's /usage now shows an approximate share of recent usage for each MCP server, and its documentation advises preferring CLI tools where available and disabling unused servers. It also warns when a single tool result passes 10,000 tokens. A chatty MCP server doesn't send you a bill; it quietly shortens your week. For an MCP vendor, that makes efficiency a retention problem: users can now see which server ate their limit.
6. Where efficiency and safety meet: partial writes
A change that takes five tool calls is a change that can stop after three. MCP has no transactions across separate tool calls (a single tool can still run an atomic transaction in its backend), so when a later step fails, the earlier ones stay done. We found this class of problem in our audit of GitHub's MCP server: a multi-file push that fails can leave behind the branch it just created (issue 3755), and under some conditions a pull-request update applies the title change and then reports an error (issue 3756).
That's why the efficient design and the safe design are usually the same design. One batched, atomic write removes turns and partial states together. We don't estimate this risk from a general failure rate; we measure it. We inject failures at each step of a multi-step write and count the partial-state rate: the share of runs that end in neither the before-state nor the intended after-state. Recovery and retry safety are reported separately.
The metrics we use instead of "tokens"
| Metric | What it measures | Why it matters |
|---|---|---|
| Cost per verified result | Total cost of all attempts, failures included, divided by the number of attempts whose end state was independently verified as meeting the task's acceptance criteria; reported with the success rate | In SWE-Effi, failed attempts tended to cost more than successes (over 4× for SWE-Agent with GPT-4o-mini); this prices them in |
| Sequential model cycles | Median and 90th-percentile model inference cycles per task; tool calls and batches counted separately | Each cycle is another chance to go wrong, and usually most of the waiting |
| Wall-clock time | Median and 90th-percentile time to a finished task | Fewer cycles don't map one-to-one to less time, so we time it |
| First-call accuracy | How often the agent picks the right tool with valid arguments first time | The most common failure, and the one design fixes |
| Crowded-room accuracy | Success with 5–10 other common servers installed alongside | How people actually run agents, and rarely measured |
| Partial-state rate | Share of fault-injected multi-step writes that end in neither the before-state nor the after-state | Ties efficiency to data safety |
| Plan-limit share | The server's approximate share of the user's recent plan usage, as the client attributes it | Users can now see it |
Our Audit + Agent Benchmark package already measures cost and turns per task on real jobs with verified end states; we plan to add the rest once the study below has run.
When efficiency doesn't matter much
If a server has a handful of tools, gets called a few times a week, returns compact results and runs in a client that loads definitions lazily, optimising it further won't change much. The effort pays where an agent uses the server every day, writes through it, or runs it alongside several others.
A checklist for MCP server designers
- Batch reads: N items in one call, with a sensible limit.
- Writes return the ID and URL of what they created, so the agent never searches for something it just made.
- Errors say what to do next, not only what went wrong.
- Pagination with a clear signal that more results exist.
- Preconditions on writes, so an update only changes what was read, and idempotency keys so a retry applies once.
- Atomic multi-step writes where the backend supports them, and exact partial-failure reports where it doesn't.
- Descriptions that say what the tool is for and when to call it.
We build to the same list: the rules our MCP server builds follow.
What isn't proven yet, and the study we plan to run
Here's the gap. Existing research measures tool-description quality, tool discovery and agent performance on specific benchmarks. What we haven't found is a reproducible study that changes one MCP server design decision at a time and measures the effect across several models on real engineering workflows, with verified end states and wall-clock time. The evidence above is real but partial: a five-query vendor case study, single-model tests, internal evaluations without published methods. Nothing yet shows agents prefer tools because they are easier to use.
So we plan to run one, in the open. Nothing is registered or run yet; this is the plan:
- Hypotheses will be registered before any runs, and published so nobody can say we moved the goalposts, and so we report the result whichever way it goes.
- The same representative engineering tasks (fix a bug through to a merged PR, review a PR, triage issues, a multi-file refactor) in disposable test repositories, to be run against GitHub's official MCP server and against a version with targeted fixes, each fix also tested on its own.
- Several models, a run count per condition fixed in the pre-registration, randomised run order, a fresh session and identical repository state for every run, and a crowded-room condition with other common servers installed. Runs will be scored by deterministic checks of the end state, not by an LLM judge.
- Everything will be published: harness, data, sanitised transcripts and failed runs.
Sources
- Chroma: Context rot (Jul 2025)
- Anthropic: Advanced tool use (Nov 2025)
- Anthropic: Effective context engineering for AI agents (Sep 2025)
- MCP-Universe, arXiv 2508.14704
- OpenAI function-calling guide
- LiveMCP-101, arXiv 2508.15760
- Hasan et al., MCP tool description smells, arXiv 2602.14878
- Agentic coding in the wild (Copilot telemetry), arXiv 2608.00101
- SWE-Effi, arXiv 2509.09853
- Vercel: We removed 80% of our agent's tools (Dec 2025)
- Speakeasy: dynamic toolsets
- Claude Code: costs · Claude Code: MCP
- Anthropic prompt caching · rate limits
- Our own measurements: the IsoKron GitHub MCP server audit and upstream issues 3751, 3747, 3748, 3755, 3756.