Self-funded research by IsoKron, conducted and written with AI assistance. Not commissioned or endorsed by PostHog; IsoKron is not affiliated with PostHog. IsoKron sells MCP server audits.
We audited PostHog's MCP server on 4 and 5 November 2026: the services/mcp code in PostHog's main repository at commit 450124f0, and the hosted server at mcp.posthog.com, used with test projects in our own PostHog Cloud account. On 10 November we re-checked every finding against master at commit c7e9544d. This article lists the findings, the fixes and their upstream status.
The finding with the strongest evidence is a lost update. Two calls that changed different fields of the same insight at the same moment both returned success, and one change was gone: in 7 of 11 pairs through the hosted MCP server, and in 6 of 11 when the same PATCH requests went straight to PostHog's API. The loss was in PostHog's API, not in the MCP layer. A PostHog maintainer approved our fix the next morning and merged a hardened version of it about 17 hours after we opened ours. Cohorts and dashboards lost writes in the same tests and still show the same code pattern on master.
The audit tracker holds 114 rows, of which 29 are findings. The second refuter rated none above medium, and most are low. Fifteen fixes exist so far: one merged, three open, and eleven ready. The 10 November re-check read code and ran local tests only; afterwards one finding (V6) was reproduced live before its fix was filed. No before/after benchmark was run.
Setup and boundary
All live calls went to projects in a PostHog Cloud account IsoKron owns. The first working session ran in one project; every scripted probe and later live call went to a second, dedicated test project with a key restricted to it. No other customer's data was touched.
| Item | Original audit, 4 to 5 November | Re-verification, 10 November |
|---|---|---|
| Code | PostHog/posthog at commit 450124f0: services/mcp and the product tool definitions, 1,193 tools | master at commit c7e9544d; services/mcp has 179 commits since the audit commit |
| Server | Hosted server at mcp.posthog.com, plus in-process checks against a local build of the audit commit | Local only: a sparse checkout and its unit tests |
| Target | Two projects in one PostHog Cloud US account: the first working session in one, every scripted probe in a dedicated test project reached with a key restricted to it | None during the re-check; the V1 and V6 checks described below came afterwards |
| Method | Scripted probes held to 2 requests a second; in-process checks with the server's real schemas and handlers; one agent session | Code reading; a local test or check where possible; 13 fix branches |
| Live writes | Test objects in that project; the probes were set to delete what they create | 0 during the re-check; one comment created and deleted afterwards for V1 |
Two more live runs sit outside those passes. On 8 November a short race probe went straight to PostHog's REST API in the dedicated test project. On 10 November, after the re-check, the V6 and V1 checks made 8 calls through local builds of the server and 8 REST requests in that project, including one comment created and deleted.
Not tested: other tenants or any attempt to cross a tenant boundary; emails, invites, exports to third parties, integrations and paid features; load; the EU region and self-hosted PostHog. The hosted server's deployed commit could not be pinned, so live results are tied to their dates, not to a commit.
Headline finding: concurrent PATCHes drop a write (C1)
Two callers change different fields of one object at nearly the same moment: A sets name, B sets description. Both get a success response showing their own change. One change is then missing from the stored object, and neither caller is told.
| Object | Through the hosted MCP server, audit | Straight to PostHog's API, 8 November |
|---|---|---|
| Insight | 7 of 11 pairs lost a write | 6 of 11 |
| Cohort | 3 of 11 | 3 of 11, then 11 of 11 in a second run |
| Dashboard | 1 of 11 | 6 of 11 in each of two runs |
| Annotation | 0 of 11 | not run |
| Action | 0 of 11 | not run |
| Feature flag (row lock) | 0 of 11 | not run |
| Notebook, title against content (lock and version check) | 0 of 1 | not run |
- Where the loss is. For these fields the MCP tool sends only what the caller gave. PostHog's update code loaded the row without a lock, applied the change to that copy and saved every column, so a write that landed in between was overwritten with its old value. The direct runs have no MCP server in the path and lose writes all the same. This is the reverse of our GitHub audit, where the MCP server's own read-merge-write introduced the loss and GitHub's per-field API lost 0 of 28 pairs.
- Controls. Feature flags, which lock the row, lost 0 of 11 pairs. Two different-field writes sent one after the other, the second from a stale read, lost nothing. Annotations and actions share the code path by reading and lost 0 of 11.
- What the counts mean. The pairs were sent within a few milliseconds of each other on purpose, and cohorts gave 3 and then 11 of 11 a minute apart. The numbers show that the race exists; they are not a rate for normal use.
- Fix. We opened pull request 131519 on 8 November: lock and reload the row before applying the request, as PostHog's feature flag code already does. A maintainer approved it the next morning. Fork pull requests cannot run PostHog's backend CI, so the maintainer re-created it as pull request 131620, narrowed the lock, handled an insight deleted mid-request and merged it on 9 November. We have not re-run the race since; the evidence for the fix is the regression test in that pull request.
- Still open. Cohorts and dashboards are unchanged on master, by code reading on 10 November. A follow-up is planned after a small live re-check. Until then, teams running several agents against one project should have one writer at a time update them, or read the object back after writing.
Other findings by class
Many of the other findings sit beside a fix PostHog had already made: the earlier change covered one code path and a sibling path was missed.
- Wrong writes (G1, X1): values the caller did not send, or did not mean, reach the API. Both changed stored values in the audit.
- Results that look complete but are not (V6, V5, Q5, K1, J1): a partial page with no next-page hint, a list that ignores a project default, missing search notes and keywords, and an unreadable body returned as success.
- Valid calls rejected (V1, X2, X3, X4): one tool rejects the real ids it is given, two
execparsing faults report a tool that exists as unknown, and a third throws an untyped error. - Server errors where a validation error is due (F4, L3, R10).
- Guidance and tool metadata that contradict the system (K3, P4, L2, N2; L1, R6, L4).
- Response content (R2, V2, V4, R3): more than a sibling tool, or the server's own rule, would return.
- Session state (N1) and repository hygiene (G3, V8).
Findings register
All 29 findings are listed. Severity is the second refuter's rating. Evidence is the state on master on 10 November:
- Test: a new unit test fails on master and passes with our fix.
- Check: a local executable check shows the behaviour; no fix was written.
- Code: code reading only.
- Not re-checked: the behaviour is in the hosted backend, and that pass made no live call.
"Live in audit" means observed against PostHog Cloud on 4 or 5 November 2026; the day is given where the tracker records it.
| ID | Finding | Where the cause is | Class, severity | Evidence | Upstream status |
|---|---|---|---|---|---|
| C1 | Concurrent PATCHes to different fields of one object drop one write; both callers get success | Backend API | wrong write, medium | Insights: fixed, by reading the merged change. Cohorts, dashboards: Code. Live in audit and on 8 November | Insights merged in PR 131620. Cohorts and dashboards: not yet filed |
| G1 | PATCH tools add nested schema defaults the caller did not send | MCP codegen | wrong write, medium | Check. Live in audit on three tools | Not filed; issue drafted. A fix needs PostHog's code generator |
| X1 | Generated schemas read the string "false" as true and null as 0 | MCP codegen | wrong write, medium | Code. Live in audit | Our PR 131514, open, not yet reviewed |
| V6 | List results through exec in text mode drop the next-page hint; 23 list tools that return a plain paginated response also drop the count and next link | MCP server | incomplete result, medium | Test. Live on 10 November through local builds: a 1-of-3 page had no count, next link or hint before the fix, and the footer after it | Pull request 132357, open |
| V5 | The six query-*-actors tools ignore the project's test-account default | MCP server | incomplete result, medium | Test | Fix ready; wave 2 |
| Q5 | A search match on a tool the key lacks scope for removes the truncation flag and other search notes | MCP server | incomplete result, low | Test. Live in audit: 49 of 50 searches | Fix ready; wave 3 |
| K1 | Hosted exec search ignores search_keywords; today four tools behind one feature flag | MCP server | incomplete result, low | Test | Fix ready; wave 3 |
| J1 | A 2xx response with an unreadable body is returned as a successful result | MCP server | incomplete result, low | Test | Fix ready; wave 5 |
| V1 | tasks-comments-retrieve rejects real comment ids | MCP server | rejected call, medium | Test. A real comment id from PostHog Cloud failed the tool's check before the fix and passed after it; the tool itself was not called through a running server | Pull request 132356, open |
| X2 | In exec, a body after a newline or tab is read as part of the tool name | MCP server | rejected call, low | Test | Fix ready; wave 2 |
| X3 | In exec, a JSON body that is not an object throws an untyped error | MCP server | rejected call, low | Test | Fix ready; wave 2 |
| X4 | exec info reads trailing flags and extra words as the tool name | MCP server | rejected call, low | Test | Fix ready; wave 3 |
| F4 | Schema-valid values for two fields returned HTTP 500; a third case depended on state | Backend API | server error, low | Not re-checked; declarations unchanged. Live in audit | Not filed; live re-check first |
| L3 | tags is typed as an array of anything; non-string tags returned HTTP 500 | Backend API | server error, medium | Code. Live in audit: 5 of 5 cases | PostHog's own PR 130103, closed unmerged 10 November; not filed by us |
| R10 | Eight GET endpoints returned HTTP 500 for an id that is not a UUID | Backend API | server error, low | Not re-checked. Live in audit: 8 of 8 | PostHog is fixing these per endpoint: PR 130031 merged, PR 128118 open |
| K3 | A skill says a suppression rule cannot be edited; the update tool is enabled | Skill text | guidance, low | Code | Wording fix ready; wave 4 |
| P4 | A search hint tells users to approve scopes only the server can grant | MCP server | guidance, not rated | Test | Fix ready; wave 3 |
| L2 | Two tool descriptions name tools that are not in the catalog | MCP definitions | guidance, low | Check | Not filed; low priority |
| N2 | alert-update moved next_check_at; its description says the value stays unchanged | Backend API | guidance, not rated | Not re-checked. Seen once live, 5 November | Not filed; live re-check first |
| L1 | persons-property-delete is not marked destructive | MCP definitions | tool metadata, low | Check | Not filed; a judgement call |
| R6 | Read-only mode is decided by a hint that some tools set loosely | MCP definitions | tool metadata, low | Code. Not called live | Not filed |
| L4 | Fields the server computes are advertised as inputs on ten tools | Backend API, MCP definitions | tool metadata, low | Code | Held; two earlier contributor pull requests (112291, 115571) were closed unmerged |
| R2 | external-data-sources-wizard returned about 225,000 tokens in one response | MCP definitions | response content, medium | Not checked locally. Live in audit | Covered by a maintainer's draft PR 129174 |
| V2 | user-settings-update returns the whole user object; user-get returns a trimmed one | MCP definitions | response content, low | Test, mocked API | Fix ready; wave 4 |
| V4 | logs-group-by is not covered by the Logs redaction hook | MCP definitions | response content, medium | Code | Not filed |
| R3 | 403 and 429 error text includes the upstream API host | MCP server | response content, low | Test. Seen on hosted in audit | Fix ready; wave 4 |
| N1 | The active project switched to a project in another of the same user's organizations | MCP session | session state, not rated | Not re-checked. Seen once live, 5 November, after a second organization had been created in the web UI | Not filed; live re-check first |
| G3 | A cleanup regex in the code generator matches nothing | MCP codegen | hygiene, low | Code; rests on the audit | Not filed; low priority |
| V8 | 62 schema snapshot files are never checked by the tests | MCP tests | hygiene, low | Code | Not filed; low priority |
Statuses checked 10 November 2026; dates are UTC. Ids are the audit tracker's row ids, and gaps in the numbering are tracker rows that are not among the 29 findings. Wave numbers are the planned order and may change. The second refuter reclassed L4 and R6 as design gaps. P4, N1 and N2 were not sent to it. The R2 figure is a cl100k estimate.
What held up
- Writes with a lock or a version check. Feature flags lost 0 of 11 pairs. In a pair of concurrent rollout changes that carried a version, exactly one caller got a conflict error that said nothing had been written. The general
update-feature-flagtool is different: on older flags it accepts a version and ignores it, so two versioned edits both returned success and one replaced the other. The tracker classes that as a design gap, so it is not among the 29. - The MCP layer in the race.
insight-updatesent only the fields it was given. - Coercion is bounded.
filterTestAccounts: "false"and a nullaggregation_group_type_indexdid not flip oninsight-create; only fields generated as coercing types did. - Cut-off responses. In local fault injection, a response cut mid-stream was reported as a transport error, not as success.
- Explicit values.
filterTestAccounts: true, sent explicitly, was honoured by both the trends tool and its actors tool. - Our own candidates. 21 of the 114 rows closed as intended behaviour, not reproduced, or our own error.
How findings were checked
- First pass, 4 November: a working session through the hosted server produced 37 candidates. A first refuter left 1 as a bug, dropped 13 and moved 23 to design gaps or feature proposals.
- Deeper passes on the code generator, schemas, guidance text, the
execdispatcher, tool variants, hosted reads, search, fuzzing of create and update calls, concurrency and fault injection. - Five gates for every row: reproduction, whether the behaviour is intended, what the documentation says, earlier upstream reports, and the strongest case against.
- A second refuter on 26 of the 29 findings: 17 confirmed, 7 narrowed, 2 reclassed as design gaps.
- Re-verification against master on 10 November, with no live call, followed by a live reproduction of V6 and a check of V1 against a real comment id before those two fixes were filed.
- Reproduction by a person. The records show a person re-ran C1 on 8 November; they do not record a human re-run of the others. X1, V1 and V6 were filed on tests and agent checks: V6 was reproduced live by an AI agent on 10 November, and for V1 a real comment id was checked against the tool's schema.
The 114 tracker rows closed as 33 bugs (the 29 findings and 4 live confirmations merged into them), 39 design gaps, 10 feature proposals, 11 duplicates, 14 intended behaviours, 3 not reproduced and 4 errors of our own. Design gaps and feature proposals are not reported here.
On 10 November, 12 findings had a new test that fails on master and passes with the fix, 3 had a local check and no fix, 7 were present by code reading only (C1 among them, for cohorts and dashboards), 3 were already reported or in progress upstream (X1, L3 and R2; the register marks two of them Code because we also read the code, and the pull request for L3 has since been closed), and 4 could not be verified without a live call.
On master, 193 of 197 unit test files pass (5,503 tests); the other 4 import files outside the sparse checkout. Each fix branch has the same 4 failures and no others, and adds no type error.
What the checking caught: X4 was narrowed to the info command and L1 from five tools to one; a size figure in Q5 was in characters, not tokens; and one statement in the tracker's L4 row was wrong at audit time.
Upstream status, 10 November 2026
- Merged: the insights fix, in pull request 131620. The maintainer's version went further than ours.
- Open: pull request 131514 for X1, opened 8 November. No maintainer has reviewed it, and its CI runs wait for maintainer approval.
- Opened 10 November: pull request 132356 for V1 and pull request 132357 for V6. Their CI runs wait for maintainer approval.
- PostHog's own work: a maintainer's draft pull request 129174 bounds response sizes, which covers R2, and R10 is being fixed one endpoint at a time. For L3, a PostHog bot account opened pull request 130103 on 5 November from server errors first seen that day, during our fuzzing. A maintainer closed it on 10 November, noting that the errors all came from one fuzz harness and that a backend-only change would be enough. That is fair: the only traffic known to hit L3 was, as far as we can tell, ours.
- Not filed: the 11 other ready fixes, and the findings marked not filed in the register.
Why the fixes go up in waves
PostHog publishes an AI contributions policy. It closes AI-assisted contributions that do not meet its bar and blocks an account after two such closures. It also says: "Prefer PRs over AI-generated issues." PostHog's contributing guide explains that fork pull requests get no backend CI run, so after review a PostHog engineer re-creates each change; on our three open pull requests the other CI runs also wait for a maintainer's approval. Every pull request we open costs a maintainer time.
The policy is reasonable, and the pace is our choice. The fixes go up two or three at a time, smallest and most useful first, and each wave waits until the previous one has had a maintainer response. Reviewers are not handed thirteen pull requests at once.
No benchmark
No before/after agent benchmark was run for PostHog. PostHog's terms ask for written consent before benchmark results are published, and we have not yet requested it.
Limitations
- Only V6 was reproduced live on 10 November, and for V1 a real comment id was checked against the tool's schema. All other live observations date from 4 and 5 November and from the 8 November probe; F4, N1, N2 and R10 may have changed since.
- The race counts are small, timing-dependent and deliberately near-simultaneous. N1 and N2 each rest on a single live observation.
- The insights fix was verified by reading the merged pull request, not by a live re-run.
- The re-verification used a sparse checkout. Its type-check reports 242 errors on master for files that are not checked out; "no new type errors" means a fix branch adds none. A full type-check was not run.
- No backend test, no MCP integration test and no code generation could be run. That is why G1 and the rest of C1 have no patch, why G3 rests on the audit, and why V2 needs one generated file regenerated before it is filed.
- Node 25.8.1 was used where the repository asks for Node 26; no test failure was traced to this.
- The findings were found with AI assistance. The model that ran the refuters was not recorded, so we do not claim it differed from the finder's. The records show a human re-run only for C1.
- One account, two projects and one region. Token figures are cl100k estimates, not Claude's tokenizer.
About this research
IsoKron performs fixed-price MCP server audits with the same method: probes against a pinned build, independent refutation, re-verification before filing and fixes sent upstream. The first public audit is of GitHub's MCP server. Details: MCP server audits.