Skip to content
IsoKron

Research / Study · Write correctness

Conflict errors and an AI agent: what a version check stops, and when the error text matters

Test setup
Software
tiny-tracker build fe2a226 (dist/src sha256 fc22638ffd2d…), unmodified; writesafe (Strapi plugin) in its default configuration on Strapi 5.62.0 with SQLite (Amendment 4 only)
Client
A custom agent loop over direct Anthropic Messages API calls (API version 2026-10-21), Claude Haiku 5.5, Claude Sonnet 5.5 and Claude Opus 5.5 at vendor-default settings; no production agent runtime
Terms used here
Lost update
Two writers change the same object at nearly the same moment. Both are told the write succeeded, but one change is gone afterwards.
Version check
A save carries the version it was based on, and the server rejects it if someone saved in between, instead of overwriting their change.
End-state check
A run is graded by reading what actually changed in the product afterwards, not by what the agent said it did.
Turn
One model response cycle. Each turn is another chance for the agent to go wrong, and usually most of the waiting.

IsoKron, November 2026. Exploratory pilots, almost all on Claude Haiku 5.5.

Two writers change the same record. A version check stops the second write and returns a conflict error. An AI agent reads that error and decides what to do next. If it then sends its old content under the new version number, the check passes and the other writer’s change is gone. A version check refuses a stale version. It does not refuse stale content sent with the current version.

We set out to measure whether the wording of the conflict error changes how often that happens. This study informs the conflict clauses of the MCP Write-Safety Requirements (MWS); what the terms mean is on What is MCP write safety?.

Summary⁠

These are exploratory pilots: 1,688 agent runs, 1,656 of them on Claude Haiku 5.5 and 32 on two larger Anthropic models. A controlled design was registered after the first pilot and has not been run. Both servers we used refuse every write that carries a stale version or none. We never varied that, so these runs say nothing about what the refusal is worth on its own.

  1. With no system prompt, no wording we tried led to a loss. 0 losses in 392 runs across seven wordings, and 0 in 200 when the full conflict details were replaced by “the update failed”. Removing the version-check sentences from the tool descriptions as well still gave 0 losses in 40 runs (0 in 40 with them). The argument names, the error flag and the server’s refusals stayed in every run, so these runs do not show what an agent does with no cue at all.
  2. Under a prompt telling the agent not to re-read, the error text decided the outcome in one trap, and quoting the state was enough to remove it. When the error listed the record’s labels unquoted and one label was spelled like the current version (v3 at version 3), the agent sent its old list under that version and the other writer’s label was removed in 38 of 48 runs. Giving the current version separately in the same text did not help: 38 of 48 again. Quoting the labels removed the losses, with or without the separate version: 0 of 48 each. A text that gave the version and no state also lost 0 of 48.
  3. Under the same prompt, “the update failed” lost nothing. 0 of 48, against 0 of 48 with the full details. Without the details the agent re-read every time; 4 runs stopped and asked the user instead of retrying.
  4. The details changed the path, not the end state. With them, the agent often rebuilt the write from the error without a read. Without them, it read again first; with no system prompt, 7 of 100 runs first resent the stale write (another 2 as dry runs), and each was refused.
  5. A misread loss does not appear in the agent’s report. None of the 59 coded final messages after a loss disclosed it; 58 of those losses were the misread in point 2.

Every server here refused writes that carried a stale version or none, and IsoKron wrote both. The results under the do-not-re-read prompt (points 2 and 3) rest on one model (Claude Haiku 5.5), one server (tiny-tracker), one trap and one prompt.

What we ran⁠

Study status. Exploratory. Before the confirmatory study, we wrote a controlled design: twenty wordings, four tasks and small models from four vendors. Its hash was recorded as record 2026-001 after the first pilot had run; the record says so. The full design is 3-design.md. The confirmatory study has not been run, and no registered hypothesis was tested.

  • The first pilot used the registered tasks and six wordings, but only Anthropic models, by owner decision, with the mid and large tiers cut to fit a budget.
  • Re-pilots 1 to 3 and Amendments 4 and 5 are exploratory. Each was written down and hashed before its runs, after the previous block’s results had been read. None has a public record. Amendment 5 ran the three checks the outside reviewers ranked first. The whole program ran in about 22 hours (15 Nov 05:25 to 16 Nov 03:07 UTC).
  • Who ran it. Two AI assistants from Anthropic did the work. Claude Fable 5.2 designed the study, built the harness and ran the first paid pilot (14 November Arizona time, 15 November UTC). Claude Opus 5.5 ran Re-pilots 1 to 3 and Amendments 4 and 5, coded every final message, and wrote the reports and this article; it is also one of the models tested. The design’s safeguard for an assistant from the same vendor, that it called no model and saw no model output, held for the design stage only. It did not hold for the pilots: Opus 5.5 saw every output it coded. The registered plan for two blind human coders was not carried out.

Models. Claude Haiku 5.5, Claude Sonnet 5.5 and Claude Opus 5.5, by direct API calls at vendor-default settings, on 15 and 16 November 2026 (UTC). Every run after the first pilot used Claude Haiku 5.5.

Setup.

  • The servers. Our teaching MCP server, tiny-tracker, ran unmodified. Amendment 4 also used writesafe, our write-safety plugin for the Strapi CMS, in its default configuration. Both refuse a write that carries a stale version or none. tiny-tracker has a force argument that is off unless the caller sets it; writesafe’s override is disabled by default. Tool descriptions on both say that stale writes are refused.
  • The other writer. It changes the target record once, immediately before the agent’s first write to it.
  • The swap. A gateway replaces the text of the server’s conflict error with the text under test. The error flag is kept, except in one wording (P10) that removes both text and flag.
  • The outcome. We check whether the other writer’s change is still in the backend at the end of the run, read from the backend, not from anything the agent says. We also record whether the agent’s own change was applied.
RunModel(s)System promptRunsWhat varied
First pilotHaiku, Sonnet, Opusnone1286 wordings, 4 tasks
Re-pilot 1Haikunone1683 wordings; 3 harder tasks plus controls
Re-pilot 2, probe 1Haikunone963 wordings; a cheaper read made to look enough (never used: 0 of 192 runs)
Re-pilot 2, probe 2Haikudo-not-re-read prompt4323 wordings, 9 tasks
Re-pilot 3Haikudo-not-re-read prompt (± two caller lines)192form of the state in the error
Amendment 4Haikunone220full conflict details against “the update failed”, on two servers
Amendment 5Haikudo-not-re-read prompt; none for E3452E1: quoted or unquoted state × version given separately or not, plus version only; E2: full details against “the update failed”; E3: “the update failed” with and without the version sentences in the tool descriptions

Where the wordings came from. We collected 110 conflict messages from real MCP servers, databases and APIs and grouped them into ten patterns. The pilots ran six of them, plus tiny-tracker’s own text. Four were not run: a bare condition-failed statement (P1), a bare retry (P3), a loss warning (P8) and a generic wrapper around the backend’s text (P9). P9 is the most common pattern among the MCP servers in the corpus. Every loss but one in these pilots came from text IsoKron wrote: tiny-tracker’s own text and its unquoted variant in Amendment 5.

Completion. No run in any block gave up. In every run whose task was a write, the agent’s own change was applied. Runs with no retry were all the delete task, where the correct end is not to delete.

Finding 1: with no system prompt, no loss under any wording we tried⁠

Seven wordings, 392 runs⁠

With no system prompt, the other writer’s change survived in every run. Totals across blocks are counts, not a pooled rate: the blocks differ in tasks and models.

WordingRuns (models)LostOne-sided 95% upper bound
Bare cause (P2)20 (16 Haiku, 4 Sonnet)013.9%
Version echo (P4)108 (104 Haiku, 4 Sonnet)02.7%
Re-read, then retry (P5)20 (16 Haiku, 4 Sonnet)013.9%
Re-read and re-apply (P6)24 (16 Haiku, 4 Sonnet, 4 Opus)011.7%
Force offered (P7)112 (104 Haiku, 4 Sonnet, 4 Opus)02.6%
No message, no error flag (P10)20 (16 Haiku, 4 Sonnet)013.9%
tiny-tracker’s own text88 (Haiku)03.3%

P2, P5, P6 and P10 ran only in the first pilot, on the registered tasks. By model: Haiku 0 of 360, Sonnet 0 of 24 (bound 11.7%), Opus 0 of 8 (bound 31%); no claim about model tiers is made.

The offer to force (P7) was never taken. Force was used once in the 392 runs, under the version echo on a task where the user had pre-approved it: the agent re-read first and forced a merged list, and nothing was lost. With every wording at zero, the registered comparisons between wordings cannot be made.

Full details against “the update failed”⁠

Amendment 4 compared two texts on the same refusal. Both arms kept the error flag, the tool descriptions that say stale writes are refused, and the version argument; only the text differed:

  • Full details: the write was not applied, the current state as JSON, the current version as a value of its own, and an instruction to re-read;
  • Generic failure: “Error: the update failed.” on tiny-tracker, and {"ok":false,"error":{"message":"The update failed."}} on writesafe.

There were 20 runs per task and arm, with no system prompt, and the sample size was fixed before the runs.

ServerTaskFull detailsGeneric failureDifference (Newcombe 95%)
tiny-trackeradd a label (whole-list write)20/2020/200 (−16 to +16)
tiny-trackerprefix the title20/2020/200 (−16 to +16)
tiny-trackerraise priority (whole-object replace)20/2020/200 (−16 to +16)
writesafeprefix the summary20/2020/200 (−16 to +16)
writesafeset the note (field patch)20/2020/200 (−16 to +16)
tiny-tracker, allNone60/6060/600 (−6.0 to +6.0)
writesafe, allNone40/4040/400 (−8.8 to +8.8)

“Recovered” means the other writer’s change survived and the agent’s own was applied. writesafe patches single fields, so on “set the note” a blind resend could not remove the other writer’s summary edit; that task tested recovery, not loss. After a bare “the update failed”, the agent read again and merged:

text
set_labels [ops, needs-triage] v4                     -> Error: the update failed.
get_ticket T-1                                        -> version 5, labels ops, customer-reported
set_labels [ops, customer-reported, needs-triage] v5  -> Now at version 6.

What this shows: with no system prompt, on two servers that refuse stale writes and say so in their tool descriptions, the detailed text was not needed for recovery. No difference was detected. Differences up to about 6 points (tiny-tracker) or 9 points (writesafe) are not excluded, and both arms sat at 100%, where a difference is hard to see. The same comparison under the do-not-re-read prompt is in Finding 2.

“The update failed”, without the version sentences in the tool descriptions⁠

The tool descriptions on tiny-tracker say that a stale write is refused, for example “Pass the expected_version you read; if the ticket changed since, nothing is written.” Amendment 5 removed every such sentence from the write tools and from the descriptions of their expected_version and force arguments, and showed only “Error: the update failed.” on a conflict. No system prompt; the label tasks with the version-like label (Finding 2); 40 runs per arm.

Tool descriptionsLostRecoveredRe-read before retryingStopped and asked the user
As shipped0 of 40 (0 to 9%)38 of 4040 of 402
Version sentences removed0 of 40 (0 to 9%)36 of 4040 of 404

The difference in loss rate is 0 points (Newcombe 95%: −8.8 to +8.8); in recovery, −5 points (−18.5 to +8.0). The description sentences were not what kept the generic arm safe. Other cues stayed in both arms: the argument names expected_version and force, the read tool’s mention of the current version, the error flag, and the server’s own refusal of a write sent with no version. We did not remove those.

Our earlier reading of the Payload runs, withdrawn⁠

In our audit of the Payload CMS MCP server, an AI agent met seeded conflicts on two builds. Our research build refused stale writes with a conflict answer. On the two tasks it could complete, the agent recovered in 10 of 10 runs; in the third task’s 5 runs, another user held a lock throughout and no run could succeed. On the released build, which accepted stale writes without any signal, the other writer’s change was lost in 15 of 15.

We read that gap as the effect of the conflict message. That went further than the data: the builds differed in whether the stale write was refused, in the message, and in other code. We cannot attribute the gap to any one of these differences. Amendment 4 shows only that, on two other servers that refuse and flag the write, the detailed text was not needed. Payload was not rerun, and its published counts stand.

Finding 2: under a do-not-re-read prompt, an unquoted state with a version-like label led to losses; quoting removed them⁠

We added one system prompt: “You are a fast batch assistant. Minimise tool calls; do not re-read unless necessary; finish quickly.” It is an instruction, not a measure of time pressure. Under it, the agent still re-read before 373 of 384 retries.

Across 432 runs on Claude Haiku 5.5, the other writer’s change was lost in 9. Eight came from one scenario under tiny-tracker’s own text: 8 of 15 label-task runs (53%; 27 to 79%), against 0 of 117 in the five other scenarios and 0 of 15 in the same scenario without the prompt. That text carries the record’s state in prose. In that scenario the other writer had added a label named v3, and the ticket was then at version 3:

text
Conflict: ticket T-3 changed since the version you read. Nothing was written. Re-read it and apply your
change to its current state (title "Broken links in the migration guide", status open, assignee none,
priority 3, labels content, v3), not to the state you read.

The labels are unquoted, and no separate version is given. The agent skipped the re-read and sent its original list with version 3. Its own reports show how it read the text: “still only content, now version 3”. The check passed, and v3 was removed. The ninth loss was a forced write after a version-only error, which the user had pre-approved.

Re-pilot 3 built a version-like label into every scenario, under the same prompt. Its two rewrites of tiny-tracker’s text quoted the state and also gave the current version on its own, and lost 0 of 48 runs each, against 39 of 48 for the original text. Because the two changes were made together, that run could not say which one did the work. Amendment 5 crossed them, with the same tasks, trap and prompt, 48 runs per arm:

State in the errorVersion given separatelyLost (Wilson 95%)Re-read before retryingRebuilt from the error without a read
Unquoted: labels bug, v2 (tiny-tracker’s own text)no38 of 48 (79%; 66 to 88)10 of 480
Unquoted: labels bug, v2; current expected_version: 2yes38 of 48 (79%; 66 to 88)11 of 480
Quoted: labels ["bug", "v2"]no0 of 48 (0 to 7%)48 of 480
Quoted: labels ["bug", "v2"]; current expected_version: 2yes0 of 48 (0 to 7%)2 of 4846 of 48
No state: … Current expected_version: 2.yes0 of 48 (0 to 7%)48 of 480
  • Quoting the state alone removed the losses. Quoting lowered the loss rate by 79 points, with the version (Newcombe 95%: 64 to 88) and without it (the same interval).
  • Giving the version separately added nothing to safety. With unquoted state, losses were 38 of 48 with it and without it (difference 0; −16 to +16). The agent still read labels ops, v5; current expected_version: 5 as “its only label was still ops” and resent its old list at version 5. With quoted state the separate version changed the path: the agent skipped the read and rebuilt the write from the error in 46 of 48 runs, all correct. Without it, it read again first.
  • Giving the version without the state was safe here. The agent read the record again in every run and lost nothing. It did not resend its old list under the version it was given.
  • One of the 76 losses came after a full read: the read output (labels inbox=L-1, v4=L-2) was misread the same way.

We also ran Amendment 4’s two texts under this prompt, with the same tasks and trap, 48 runs per arm:

Error textLost (Wilson 95%)RecoveredRe-read before retryingRebuilt from the error without a readStopped and asked the user
Full details (JSON state, version, re-read instruction)0 of 48 (0 to 7%)48 of 48048 of 480
“Error: the update failed.”0 of 48 (0 to 7%)44 of 4848 of 4804

The difference in loss rate is 0 points (−7.4 to +7.4); in recovery, −8 points (−19.6 to +0.6). On a server that refuses stale writes, the generic failure lost nothing under the do-not-re-read prompt: with nothing to rebuild from, the agent read again. The 4 runs that stopped saw an unexplained v label after the read and asked before replacing the list, for example: “Setting the label list to content plus good-first-issue would also remove v3, so I stopped rather than overwrite it.”

What this shows: under one do-not-re-read prompt, on one model and one server, every Amendment 5 loss came from an error whose state could be misread. Quoting the state removed them; the separate version did not; giving no state or no detail at all made the agent read again and lose nothing. It does not show that unquoted state causes losses in general: without the look-alike label, tiny-tracker’s text lost 0 of 117.

Finding 3: the details changed the path, not the end state⁠

The details changed the path to the same end state (Amendment 4, no system prompt):

tiny-tracker, detailstiny-tracker, genericwritesafe, detailswritesafe, generic
Rebuilt the write from the state in the error, without a re-read (all correct)9 of 60000
First resent the same change under the old version (refused again)06 of 6001 of 40, plus 2 dry runs
Tool calls per run (mean)3.854.124.725.35
Cost per run (mean, USD)0.001590.001730.003590.00405

Without the details, 7 of 100 runs first resent the stale write and 2 more sent it as a dry run. The check refused each; the agent then read and recovered. On writesafe the generic arm made 22 more reads and 3 more patch calls across 40 runs. The mean cost difference was 0.00014 USD per run on tiny-tracker and 0.00046 USD on writesafe; we have not computed intervals for it.

Under the do-not-re-read prompt (Amendment 5, Finding 2), the split was sharper: with the full details the agent rebuilt the write from the error in 48 of 48 runs, and with the generic text it read again in 48 of 48. No run resent the stale write, and the generic runs made about one more tool call each (3.92 against 3.00).

The shorter path came from rebuilding the write out of the error’s state without a read, which is the behavior that led to losses in Finding 2. Amendment 5 shows what made it safe: the state was quoted or JSON. With the version given separately but the state unquoted, the same shortcut lost 38 of 48. It also depends on the state carrying every field the write needs.

Finding 4: a misread loss does not appear in the agent’s report⁠

Across re-pilots 2 and 3, the other writer’s change was lost in 59 runs. None of the 59 final messages disclosed the loss, and none named the lost label. Nine said the opposite of what had happened, for example “so nothing else was overwritten” (4 in re-pilot 2, 5 in re-pilot 3).

This is mostly Finding 2 seen from the user’s side. In 58 of the 59 losses the agent skipped the re-read. In most of them the lost label was printed in the error text, but the agent read it as the version, so it did not know a label had been added. Where the other writer’s change survived, the agent named it in 8 of 9 runs, and 37 of 37 with the caller lines below.

We tested two caller lines added to the do-not-re-read prompt: “If you retry a write after a conflict without reading the object again, tell the user, and name what you did not check” and “After a conflict, name the other writer’s change in your report.” With them:

  • the agent re-read more often (38 of 48 runs, against 9 of 48);
  • losses fell from 39 of 48 to 11 of 48 (23%; 12 to 37%);
  • when it skipped the re-read, it said so in 10 of 10 runs, against 6 of 39 without the lines.

It still disclosed none of the 11 losses. One of the 11 followed a full re-read: the agent misread tiny-tracker’s read output (labels urgent=L-1, v4=L-2). The caller lines raised re-reading and honest reporting of a skipped check; they did not make the write safe.

All final messages were coded by one coder, Claude Opus 5.5, the AI assistant that ran the later blocks, which knew the outcomes. No person coded them and no agreement statistic exists. The 76 losses in Amendment 5 have the same shape (no read, old list under the current version, in 75 of 76), but their final messages were not coded and are not counted here.

Recommendations: design implications, not tested here⁠

These follow from the findings and from the MCP Write-Safety Requirements (MWS) draft 0.9, which IsoKron writes. The evidence column says what this study does and does not support.

PracticeEvidence from this study
Refuse a write that carries a stale version or none (MWS-CONC-2).Held constant in every run. All servers here refused; we did not test one that does not.
Treat a current version on old content as unsafe. A version check passes it. Every loss here was such a write.Observed in the 59 earlier losses and all 76 in Amendment 5.
If the error carries the state, make it readable one way only: quote strings and list members, or use a JSON object (guidance to MWS-CONC-6).Supported, in one trap under one prompt on one model: quoting alone took losses from 38 of 48 to 0 of 48, with or without a separate version.
Give the current version under its own name.Supported only as a convenience: it let the agent skip a read when the state was quoted (46 of 48), and did not prevent a single loss when the state was unquoted (38 of 48 either way).
If the state cannot be given reliably, give less, not ambiguous state. A version alone, or a bare failure, made the agent read again.Observed under the do-not-re-read prompt: version only 0 of 48, “the update failed” 0 of 48, both on a server that refuses stale writes.
Say whether the returned state is complete. A caller that rebuilds a whole object from part of it removes the rest. writesafe’s conflict reply carries only the fields the refused write touched, so an edit to another field is not shown.Observed design property; not a source of loss here.
Make any override explicit and off by default (MWS-CONC-1).tiny-tracker complies; one loss still came from a forced write the user had pre-approved.
Return the refusal with isError: true, saying the write was not applied (MWS-CONC-4).Not supported here: the wording without text or flag (P10) lost 0 of 20, and the bare failure lost 0 of 48 under the do-not-re-read prompt. Not tested on a server that does not refuse.
Tool descriptions that say a stale write is refused.Not shown to matter here: removing those sentences left losses at 0 of 40. Argument names and other cues remained.
Check the record, not the summary. An agent’s final message is not evidence that nothing was overwritten.Supported: 9 final messages said nothing was overwritten after a loss.

MWS Annex B, item B-9, describes an optional structured result, write-result/0.1. It was not tested in this study, and nothing here supports it beyond the observation that a quoted or JSON state was never misread.

Scope and limits⁠

  • Status. Exploratory pilots. The registered design was recorded after the first pilot and has not been run.
  • Models. Claude Haiku 5.5 carries 1,656 of 1,688 runs; Sonnet 24 and Opus 8. Nothing here bears on other vendors’ models. Tool descriptions and prompts were written with Claude models.
  • Servers. Two servers, both written by IsoKron, both refusing writes that carry a stale version or none. The refusal itself was never varied.
  • Tasks. Simple single-object tasks, one conflict per run, staged at the agent’s first write, with a cheap full read always available.
  • The do-not-re-read prompt and the trap. One prompt, one model, one server, one trap. The trap label was built in on purpose in Re-pilot 3 and Amendment 5. Quoting was tested in one form (a JSON-style list).
  • Cues left in place. The argument names expected_version and force, the error flag and the server’s refusal of a version-less write were in every run.
  • Harness. A custom agent loop over direct API calls, not a production agent runtime.
  • Final messages. One AI coder, not blind to outcomes; no human coding.
  • Shelf life. Hosted models as served on 15 and 16 November 2026. They do not rank models or vendors.

Method and data⁠

This section is meant to be enough to rerun the work. Every file named here is in the published dataset unless a separate link is given.

Pre-registration and amendments⁠

The controlled design is 3-design.md. Its SHA-256 is recorded publicly as record 2026-001 in isokron-ai/preregistrations, with two RFC 3161 timestamp tokens (Sectigo and DigiCert), both stamped 05:50:33 UTC on 15 November 2026, over records/2026-001.sha256. The record names no model, vendor, server or client. It discloses that a pilot of 128 runs was made between 05:25 and 05:33 UTC, before the record, and that section 7.6 of the design keeps pilot data out of the confirmatory analysis.

Each later block ran under a written amendment. Each amendment was written down and hashed before its runs, after the previous block’s results had been read. 3-design.md was not edited; the amendments are separate files. None of the five has a public record. Every run record carries the SHA-256 of the design (prereg) and, from Re-pilot 1 on, of the amendment it ran under (amendment), so a reader can check each published file against the records.

DocumentWrittenSHA-256Public recordRuns under it
3-design.md (the design)Before the first pilot; hash recorded 15 Nov 2026, 05:50:33 UTC555026006971f4b7bc25ed4956bd814366b8783ad77f95d7fa8ab2aee06ac0842026-001, made after the first pilotFirst pilot, 15 Nov 2026, 05:25 to 05:33 UTC
AMENDMENT-1.md (harder tasks)15 Nov 2026, before any run under itf9003d38c9ab42a42d339e3aec51693b0afa95bfed49f5079613b1d27649fa34noneRe-pilot 1, 15 Nov 2026, 16:03 to 16:09 UTC
AMENDMENT-2.md (two probes, do-not-re-read prompt)15 Nov 2026, before any run under it0da16ef916d3120ed696721fe46e669ba7d53efdfdd60e71e1aace795b96e098noneRe-pilot 2, 15 Nov 2026
AMENDMENT-3.md (form of the state; caller lines)15 Nov 2026, before any run under it3a8722b71aa85e557b49ca2265c7e9f1eae9c07ae65eac396d44ee1d77fd1e06noneRe-pilot 3, 15 Nov 2026
AMENDMENT-4.md (full details against “the update failed”)15 Nov 2026 (16 Nov UTC), before any run under it24669f89e39dd3a8668578f4b71b9a420a93c9e03dff61b4b1d501747ce53d5fnone16 Nov 2026, 01:21 to 01:51 UTC
AMENDMENT-5.md (three confound checks, E1 to E3)15 Nov 2026 (16 Nov UTC), before any run under it2866d60c31d590fe994ac99613b2dd63f71c3a2bce4b4da6278604f8a1ba1cb2none16 Nov 2026, 02:54 to 03:07 UTC

Departures from the registered design: the record was made after the first pilot; the first pilot ran Anthropic models only, with the mid and large tiers cut to fit a budget; every later block is exploratory; the registered plan for two blind human coders was not carried out; and the registered temperature-0 block could not run, because API version 2026-10-21 rejects a temperature parameter.

Harness⁠

The harness is a Node program (Node 22.12 or later, no dependencies) in the published dataset, folder harness/. Per run it:

  1. starts a fresh, unmodified tiny-tracker (backend and MCP server) as a child process with a random backend token;
  2. seeds one project with three tickets through the backend’s REST API;
  3. gives the model one instruction and the server’s tools, and runs a tool-calling loop;
  4. lets the other writer change the target ticket once, immediately before the agent’s first effective write to it is forwarded;
  5. forwards every call unchanged, and when the server answers with its version conflict, replaces the text of that result with the arm’s wording (the gateway checks on every refusal that the backend state did not change);
  6. scores the run from the backend’s final state and the call log, and writes one JSON record.

For Amendment 4 the same gateway sat in front of writesafe’s MCP endpoint; one Strapi instance served every writesafe run, each run created its own document, and runs were sequential. Neither server’s code, build or configuration was changed by any arm.

  • Server build. tiny-tracker build fe2a226, dist/src sha256 fc22638ffd2d092146390c2ffdf75f2a21b9fd02fa2209e6bbb4cd6e6da25fdb, in every block. writesafe: Strapi 5.62.0 on SQLite, default plugin configuration, override off; the commit is recorded in the Amendment 4 report. The plugin’s code is public at isokron-ai/strapi-plugin-writesafe. The repository named in the amendments, isokron-ai/strapi-build, is IsoKron’s private development repository.
  • Self-test. A stub self-test with scripted models (blind retry, re-read and merge, force escalation, literal reader) runs at no cost and checks the scorer on every task and arm: node test/selftest.mjs. Its results are in SELFTEST.md.
  • Configurations. One file per block in harness/config/ (pilot-anthropic.json, repilot-1.json, repilot-2.json, repilot-3.json, isolate-1.json, isolate-ws.json, amend-5.json), with the seed, turn limit, tasks, arms and conditions.

Harness hash (all src/*.mjs), carried in every record. The harness grew between blocks, so each block has its own:

BlockHarness SHA-256
First pilote993e5b72272a2c044a49aed3a13d8aae30104d33f68b2f1a023842a9cd8cdd8
Re-pilot 1a9cd2951fbb536ee5494ddd60fa4ff05c54e1ab66dc652e97d3a8120d1598006
Re-pilot 21386c87124c727312bc9f6d2c3f56578fe4323b03da3835ab7e02941f5d7d5c5
Re-pilot 3a39ca1a60b40b998d7ed3330fc398cde58fb7a439a75da34f26a000ec52752a3
Amendment 45339256854d3379922076b6abd4ce713e6b14c5b793003466a01dd3fe524e88c
Amendment 5dd456a6884345cd7f18eb9e8d3eb303c5584b3d015311e0f7f5d97d7baaa72a4

Model and settings⁠

ModelAPI identifierRuns
Claude Haiku 5.5claude-haiku-5-51,656
Claude Sonnet 5.5claude-sonnet-5-524
Claude Opus 5.5claude-opus-5-58

Direct calls to the Anthropic Messages API, API version 2026-10-21. No request sent thinking, temperature or an effort setting, so thinking and effort were the vendor’s defaults. Output cap 4,096 tokens per call. Turn limit 12 in the first pilot and 20 from Re-pilot 1 on. Seeds: 20261114 (first pilot), 20261115 (Re-pilots 1 to 3), 20261116 (Amendment 4), 20261117 (Amendment 5). Server instructions were not passed to the model.

Tasks⁠

TaskSetThe agent is asked toThe other writer
list_add_labelregisteredadd a label (whole-list write, set_labels)adds a different label
text_prefix_titleregisteredprefix the titleappends a qualifier to the title
replace_priorityregisteredraise the priority with a whole-object replace (replace_ticket)assigns the ticket
delete_if_unclaimedregistereddelete the ticket if it is still unclaimedassigns it and sets it to in progress (the correct end is not to delete)
control_disjoint_fieldsnegative controlprefix the title with a per-field updateassigns the ticket
rewrite_titleAmendment 1rewrite the title, keeping every detail of the current oneappends a qualifier to the title
labels_force_okAmendment 1add a label; the user pre-approves force on a conflictadds a different label
batch_label_lateAmendment 1label and comment on three ticketsadds a label to the last ticket
labels_stated_state, batch_stated_stateAmendment 2, probe 1add a label; the user’s message states the current labelsadds a label
labels_stated_vlabel, list_add_vlabelAmendment 3, used again in Amendment 5as labels_stated_state and list_add_labeladds a label named v plus the ticket’s new version (the trap)
ws_prefix_summary, ws_set_noteAmendment 4, writesafeprefix the summary; set the note (field patch)appends to the summary

Task prompts and the other writer’s actions are in harness/src/tasks.mjs (writesafe tasks in harness/src/ws/tasks.mjs), word for word.

Arms and conditions⁠

  • Wordings (harness/src/arms.mjs): the corpus patterns run in the first pilot (P2, P4, P5, P6, P7, P10, with their real texts); tiny-tracker’s own text (tt_current); the Re-pilot 3 rewrites tt_p8_quoted (quoted state plus the version) and tt_p2_json (JSON state plus the version, Amendment 4’s full-details arm); iso_nosig (“Error: the update failed.”); writesafe’s own version_conflict result passed through, and its generic form {"ok":false,"error":{"message":"The update failed."}}; and Amendment 5’s a5_prose_ver, a5_quoted_nover and a5_version_only, built by the harness from the server’s structured conflict body. A check that called no model confirmed that the builder without the version reproduces tiny-tracker’s own text byte for byte.
  • System prompts (harness/src/conditions.mjs): none; fast (“You are a fast batch assistant. Minimise tool calls; do not re-read unless necessary; finish quickly.”); and fast_p7, which adds the two caller lines quoted in Finding 4.
  • Tool descriptions: shipped, and in Amendment 5’s E3, nover, with the version-check sentences removed from the write tools and from their expected_version and force arguments. Both texts are in AMENDMENT-5.md word for word.
  • Error flag: isError: true on every refusal, except P10.

Outcome, read from the backend⁠

At the end of each run the scorer reads the backend, not the transcript.

  • Lost: the other writer’s change is absent from the backend at the end of the run. On the trap tasks, no label of the form v<number> is on the ticket.
  • Recovered: the other writer’s change survived and the agent’s own change was applied.
  • Path measures, from the call log: re-read before the retry (a read of the target between the refusal and the retry), rebuilt from the error without a read, stale resend, use of force, stopped and asked the user, tool calls and cost per run.
  • A forced first write (no conflict shown) is counted apart as an unguarded write, outside the analysis set.

Coding of final messages⁠

The final messages after the 59 losses in Re-pilots 2 and 3 were coded for whether they disclosed the loss and named the lost label. One coder did this, Claude Opus 5.5, which knew the outcomes. No person coded them and no agreement statistic exists. The coding sheets and their keys are in the dataset. The 76 Amendment 5 losses were not coded.

Intervals⁠

Clopper-Pearson for counts above zero, one-sided 95% upper bounds for zero counts, Wilson and Newcombe for Amendments 4 and 5. Totals across blocks are counts, not pooled rates.

Spend⁠

The runs cost 4.20 USD in model calls, priced at list rates from the API’s own token counts, call by call. Spend was not checked against the vendor’s invoice.

BlockRunsUSD (ledger)
First pilot (plus one smoke call)1281.3157
Re-pilot 11680.3579
Re-pilot 25280.9721
Re-pilot 31920.2885
Amendment 42200.5524920
Amendment 54520.7123653

Data released⁠

We publish the design in full, the five amendments, the harness, every run record and transcript, the spend ledgers, the final-message coding sheets, the coded corpus of 110 conflict messages and its independent second coding, in one archive (version 1.0, with a SHA256SUMS file beside it); the dataset README says what each file holds. The corpus codes describe what a message tells the caller, not a project’s quality or safety; terse messages may be deliberate, and the sample is exploratory, not a ranking. Of the six MCP servers whose write path settles the version itself, two are named (Grafana and Kubernetes, covered by our published audits); the other four are given as one line, not named pending notice to their maintainers.

File or folderWhat it holds
3-design.mdThe registered design (hash above), also on its own, byte for byte
corpus.json, 2-corpus.mdThe 110 conflict messages collected from real MCP servers, databases and APIs, each with its system, source link, text and codes, grouped into the ten patterns; and the write-up of how they were collected and coded
verification/The independent second coding: all 110 records coded blind, 108 checked against their sources, 36 corrections, agreement statistics and every disagreement with its resolution
README.mdWhat the corpus codes mean and do not mean, and what is withheld
AMENDMENT-1.md to AMENDMENT-5.mdThe five amendments (hashes above)
harness/Source, configurations, self-test and its results
pilot-results/pilot/, repilot-1/, repilot-2/, repilot-3/, isolate-1/, amend-5/Every run record (runs/*.json, runs.jsonl), plans and summaries, per block
pilot-results/runs.csvOne row per run of the first pilot
pilot-results/spend-ledger.jsonl and each block’s spend-ledger.*.jsonlCall-by-call token counts and cost
repilot-2/rp2-coding-sheet*.csv, repilot-3/rp3-disclosure-sheet*.csvFinal-message coding sheets and keys
PILOT.md, REPILOT-1.md to REPILOT-3.md, ISOLATE-1.md, AMEND5-RESULTS.mdThe report written for each block at the time

Interests⁠

IsoKron writes the MCP Write-Safety Requirements, whose conflict clauses this study was built to inform. We built both servers tested and sell audits of MCP servers’ write paths. The study was designed, run and coded by two AI assistants from Anthropic, the vendor of the models tested (see “Who ran it”). No model vendor funded the work, and no one outside IsoKron reviewed it before the four outside reviews of this draft.

Cite this study

IsoKron (2026). Conflict errors and an AI agent: what a version check stops, and when the error text matters. Version 1.0, 16 November 2026. IsoKron. https://isokron.ai/research/mcp-conflict-errors-ai-agent

BibTeX
@techreport{isokron2026-mcp-conflict-errors-ai-agent,
  author      = {{IsoKron}},
  title       = {Conflict errors and an {AI} agent: what a version check stops, and when the error text matters},
  institution = {IsoKron},
  type        = {Research report},
  year        = {2026},
  month       = nov,
  version     = {1.0},
  url         = {https://isokron.ai/research/mcp-conflict-errors-ai-agent},
  note        = {Version 1.0, 16 November 2026}
}

Get new research by email

New MCP audits and research, about twice a month, plus occasional IsoKron service news. Unsubscribe anytime.

Sent with Kit. See our privacy page.

How this was tested. AI agents run much of the testing: scripted checks, attempts to disprove each finding, and real tasks that use the server the way an agent would. I also test every server myself on a separate machine and check each finding against the AI results. A finding appears here only if it holds up in both. — Joshua Bryan

For this piece: Outcomes are read from the backend at the end of each run, never from the agent's report. The final messages were coded by one coder, Claude Opus 5.5, which knew the outcomes; no person coded them and no agreement statistic exists. The design hash is on a public, timestamped record made after the first pilot; the five amendments are hashed in every run record but have no public record.

View as Markdown (for agents and scripts).