Skip to content
IsoKron

Method · Version 1.0,

How we rate MCP write-safety findings

Five levels, set by what the write did, how often it happened and how much data it damaged. Whose code caused it is recorded beside the level, not taken off it.

How a rating is made

Every write-safety finding in an IsoKron audit gets the four inputs below and the scope of the damage, written on one line beside the rating so you can check it. The decision table turns them into a level. The restore path, whose code caused the failure, and whether it needs an input the API would refuse are written beside the level and do not change it. The scale is about write safety: whether an agent can leave your data wrong without anyone noticing.

  • Rate the worst class that is true. A mislabeled tool that also overwrites data is rated as the overwrite.
  • Rate what was measured. Harm that follows from the code but was not run is rated as code only.
  • Rate what the failure did to the data. Whose code caused it, and how unusual the input was, are written beside the rating, not taken off it.
  • Count a restore path only when it was tested on the audit’s fixtures, in the audit or in a dated follow-up.
  • The label a finding was filed with upstream, or rated with in an earlier article, is shown beside ours, not replaced.

The four inputs

1. Class: what the write did

The first six are write classes. IDs are from our published audits.

ClassWhat happenedSeen in
Silent lossA stored value is overwritten, reset or deleted, and the call reports success.GH-01, GF-03, PAY-06
Wrong targetThe write lands on a different object, field, branch or namespace than the call named.GF-02, K8S-20
Guard bypassA lock, version check, filter requirement, read-only mode or tool limit that should stop the write does not.GF-01, PAY-01, PAY-03, GH-08
Partial writePart of a multi-part write lands, and the answer does not say which part.K8S-01, K8S-03, GH-04
Unrequested changeThe tool writes values the caller did not send: defaults, coerced values, or changes to another object.K8S-02, PostHog G1 and X1
False successThe call reports success, but the requested change did not happen, or did not happen in full.GH-06, PAY-04, K8S-09
Mislabeled toolAn annotation or description says the tool is safer than it is: read-only, non-destructive or idempotent.GH-07, K8S-12, PostHog L1
Outside write correctnessRead results, error text, configuration, response size and hygiene. Rated here only for how they can mislead an agent. Exposures are listed, not rated (see below).PAY-07, K8S-07, PostHog V6

2. Reproduction: how often

Deterministic
Every trial, for example 3 of 3 or 30 of 30.
Majority
More than half of the trials, for example 19 of 30.
Occasional
At least once, in half of the trials or fewer.
Code only
Follows from the code but was not run live. Inferred harm counts here, not as measured.

3. Vendor API: does the platform prevent it?

Each finding is run again through the platform’s own API, the control every audit uses. The answer sets the attribution beside the level; it does not change the level.

Prevents it
The same operation through the platform’s own API, used as documented, does not fail. The MCP layer adds the failure.
Same failure
The platform’s API fails the same way, and the agent meets it through the server.
No equivalent
No API call compares: startup settings, annotations, MCP-only features, or an input the API cannot express. We show the closest request.

4. Data impact, and how far it reached

Lost
Data someone stored is gone, and no restore was confirmed by a test on the audit’s fixtures. Where the platform documents a history we did not test, it reads “lost (restore path unconfirmed)”.
Corrupted
The store holds a state nobody asked for: a wrong value, an object in the wrong place, or half of a multi-part write reported as complete.
Duplicated
An extra copy or a leftover object that someone has to clean up. The original is intact.
Recoverable
Lost or corrupted, and in the audit, or in a dated follow-up test on its own fixtures, we got the value back from the platform’s own history, as an ordinary user of the platform can. A history that is documented but was not tested does not count.
None
Nothing stored is wrong. The caller may still be told something false.

The scope goes at the end of the inputs line. It separates Critical from High.

One field
Part of one object: a field, a label, a list inside it.
One object
The whole object the call named, such as a dashboard, a VM or a file, however much it held.
Many objects
Objects the call did not name one by one: every match of a filter, everything in a folder or namespace, the children a parent takes with it.
None
Nothing stored was changed.

Decision table

Read from the top; the first row that matches gives the level.

LevelClassReproductionData impact
CriticalSilent loss, wrong target or guard bypassDeterministic, from one caller with no second writerLost or corrupted, reaching objects the call did not name one by one (every match of a filter, a folder, a namespace, a parent that wipes its children)
HighAny write classDeterministic or majorityLost or corrupted, up to one whole object, however much it held
MediumAny write classAnyRecoverable (restore confirmed by test) or duplicated; or lost or corrupted when occasional or code only
LowAnyAnyNone, but the caller is told something false, or a safety setting is off
InformationalAnyAnyNone, and nothing is misreported

A call that overwrites or empties the one object it addressed is High, however much that object held; the scope in the inputs line (one field, one object, many objects) shows how far it reached. Critical is for a single ordinary call whose loss reaches objects it did not name.

A refused call whose error message is false, such as a real id reported as invalid or an existing tool reported as unknown, counts as “the caller is told something false”: Low.

Beside the level

Three things are written next to every rating. None of them moves the level up or down.

Restore path
One of the three states above, for every loss. Where a confirmed restore would change the level, we say so: “High; Medium if the restore path is confirmed”.
Attribution
MCP-specific, platform-inherited or shared, from the vendor API control. It says whose code to fix. It does not lower the level: the data is as gone either way.
Reachability
When the failure needs an input the tool’s own schema or the platform’s API rejects as invalid, such as an out-of-range number or a wrong type, we say so. It does not lower the level.

Restore path: lost until a restore is tested

A history that is documented but untested fails in ordinary ways: a retention cap, a beta API version, a write path that skips the log. And a restore is never free: the loss was silent, so someone has to notice it first, and a version restore discards whatever was saved afterwards.

Confirmed by test
In the audit, or in a dated follow-up test on its own fixtures, we restored the lost value from the platform’s own history, with what an ordinary user can reach. Rated Recoverable, with how and when.
Documented, not tested
The platform documents a history that should hold the value, but we did not restore from it. Rated as lost, with what would need verifying.
None found
No history or restore action that an ordinary user can reach.

Tested so far, in a follow-up on 14 November 2026 on the audits’ own test repository and local Grafana instances: git history gave back, byte for byte, the files GH-02, GH-03 and GH-15 deleted or overwrote, and dashboard version history gave back the values GF-01, GF-02 and GF-19 lost, on Grafana 11.6.16 and 13.3.0 (GF-19 on 13.3.0 only, the version that serves v2 dashboards). All six rate Medium. Grafana’s versions_to_keep setting defaults to 20.

Attribution: whose code to fix

From the vendor API control, run for every finding.

MCP-specific
The platform’s own API does not fail this way, or no API call compares. The fix belongs in the MCP server.
Platform-inherited
The platform’s own API fails the same way and the server passes it on. The fix belongs in the platform; the agent still meets the failure through the server.
Shared
Both contribute: the platform fails this way in some cases, and the server removes a protection or adds cases of its own.
HighC1 · PostHog MCP server

Two concurrent PATCHes to different fields of one insight drop one write; both callers get success.

Rubric inputs
Silent loss · Majority (7 of 11 pairs through the MCP server) · Vendor API: same failure (6 of 11 straight to PostHog’s API) · Impact: lost, no restore path confirmed · Scope: one field per pair
Attribution
Platform-inherited. PostHog merged the fix for insights.
Why High
High on impact, like GH-01: in both, one caller’s change is gone and both callers were told it worked. What differs is whose code to fix, and that is what the attribution says.
Article label
wrong write, medium

C1 in the audit

One example per level

Each is a published finding. The five audit articles were written before this rubric. Each finding in them now shows this rating beside the label it was published with, which stays as it was.

CriticalPAY-03 · Payload MCP plugin 3.93.0

A delete with a where of {} matched every document and removed all 20.

Rubric inputs
Guard bypass · Deterministic, one caller (all 20 deleted in 5 of 5 runs; the same on v4 main) · Vendor API: no equivalent (REST cannot send an empty filter; it refused a missing one with 400 in 5 of 5) · Impact: lost · Scope: many objects, every document in the collection
Restore path
None found. The tasks collection this probe used had Payload’s defaults, with no trash and no versions, so the deleted documents are gone.
Attribution
MCP-specific. The control is REST, the API Payload offers to clients. In Payload’s Local API, which trusted server code uses, {} is a valid filter for every document; the plugin hands that to an agent without REST’s guard.
Why Critical
One ordinary-looking call removed documents it never named, and nothing restores them. The article rates it S2 on its own scale, after a refuter narrowed it because {} is a valid match-all in the Local API. That narrowing is about whose behavior it is, which the rubric records as attribution; the level is about what the call did to the data.
Article label
S2

PAY-03 in the audit

HighGH-01 · GitHub MCP server v2.1.0

Two concurrent issue_write updates to different issue fields revert each other.

Rubric inputs
Silent loss · Majority (19 of 30 concurrent pairs, three runs) · Vendor API: prevents it (per-field mutation lost 0 of 28) · Impact: lost (restore path unconfirmed) · Scope: one field per pair
Restore path
Documented, not tested. GitHub records issue-field changes in the issue timeline; we did not check that the lost value appears there, and there is no restore action, only setting the field again by hand. High; Medium if the restore path is confirmed.
Attribution
MCP-specific
Why High
Not Critical: it needs a second writer, and the loss stays inside the one issue each call named.
Article label
reliability

GH-01 in the audit

MediumK8S-03 · Kubernetes MCP server v0.1.2

pods_delete answers “Pod deleted successfully” when deleting the Pod’s Service is refused, and the Service stays.

Rubric inputs
Partial write · Deterministic (3 of 3; an injected fault, 3 of 3) · Vendor API: prevents it (the same Service delete straight to the API is refused with 403, and says so) · Impact: duplicated, a leftover Service someone has to clean up · Scope: one object
Attribution
MCP-specific
Why Medium
Something is left behind and the answer hides it, but nothing anyone stored is lost or changed.
Article label
Low to medium

K8S-03 in the audit

LowK8S-14 · Kubernetes MCP server v0.1.2

disabled_tools = ["pods_deleet"] starts the server with all 21 tools, without a word.

Rubric inputs
Guard bypass · Deterministic (3 server starts per case) · Vendor API: no equivalent (a startup setting) · Impact: none (no write made) · Scope: none
Attribution
MCP-specific
Why Low
A safety setting is off and nobody is told, but no data has changed yet.
Article label
Low (usability)

K8S-14 in the audit

InformationalPAY-07 · Payload MCP plugin 3.93.0

Single-document writes echo the whole stored document: a 21-byte update answered with 384 bytes.

Rubric inputs
Outside write correctness · Deterministic · Vendor API: prevents it (REST can trim the answer with select) · Impact: none · Scope: none
Attribution
MCP-specific
Why Informational
A cost item: nothing stored is wrong and nothing is misreported.
Article label
Info

PAY-07 in the audit

The rubric applied in the sample report

Exposures are listed, not rated

This scale measures one thing: whether a write did what the caller asked. Some findings are about what a tool reveals, such as a signed download URL in a result or a field a redaction rule should have covered. They change no stored data, so this scale has nothing to say about them, and a level here would look like a security verdict we did not make.

For each exposure we show “Not rated on this scale: exposure”, the label the upstream project gave it, what is exposed, to whom, and whether the platform’s own API returns the same thing. We do not add a security score of our own; for one, use the upstream project’s advisory or your own assessment. Findings about read results and error text that can mislead an agent stay on the scale: nothing stored is wrong, but the caller is told something false.

Not rated on this scale: exposureGH-09 · GitHub MCP server v2.1.0

Directory listings put tokenized private-repo download URLs into model context.

What is exposed
download URLs for private-repository files, each with a token that fetches the file without other authentication for a few minutes
To whom
the model the agent runs on, and anything that records its context
Platform API returns the same
yes, GitHub’s REST contents API returns the same URLs
Upstream label
Low, filed as hardening

GH-09 in the audit

Where the structure comes from

We borrowed from established scales and left out everything that is not about writes:

  • CVSS v4.0: five named levels, and the Recovery values (automatic, user, irrecoverable) that our restore-path rule borrows from. In CVSS, Recovery is a supplemental metric and does not change the score; here a restore confirmed by test can lower the level.
  • OWASP Risk Rating Methodology: integrity loss graded by how much is damaged, from a little corrupt data to all of it, which is why scope separates Critical from High.
  • Jepsen analyses: each issue listed beside the event it requires and the version that fixed it, and claims tested against what the system promises.

The rubric is versioned; a change moves the version and date at the top of this page. How an audit runs: how every finding is verified.

Request an audit or a build

Tell us which server and what you need. We reply to confirm the scope, the price and the dates.