BENCHMARKS
Measured performance, published method.
Two real experiments: the public MEME benchmark on memory that changes (100 cases, 694 questions), and the same AI model answering questions about the company with and without LAN Brain (20 questions with a verifiable answer). All of it reproducible, limitations stated, zero invented numbers.
Run 2026-08-23 · dataset HF meme-benchmark/MEME · public harness SeokwonJung-Jay/MEME-public
100%
Chained facts answered correctly (Cascade), where raw memory scores 1%
92%
Overall on MEME across the 694 questions, where raw memory scores 35%
7x
More correct answers about company data with LAN Brain (70% vs 10%)
80%
Of answers carry a verifiable citation to the source page (without the brain: 0%)
EXPERIMENT 1 · MEME
Memory that changes: searching versus descending.
MEME measures what almost no AI memory solves: facts that get updated, chained and deleted. We ran all 694 questions of the 100 cases with the same retriever on both arms, so the only thing that changes is how the content is stored and how it is read. The LAN Brain pages were written by an ingest simulator that detected the changes on its own, without ever seeing the answers, and they are read the way the product reads them: descending from the index that lists the entities of an area with their value in force down to each page.
Raw memory (BM25 over the sessions)LAN Brain: descent with an index (how the product operates)
- Cascadefacts that depend on another one that changed
- Absencesaying “uncertain” when something expired
- Trackingthe history of one fact, in order
- Deletiondeleted content stops showing up
- Aggregationseveral facts at once
- Exact Recallstatic facts · tie
Without the index, the same pages score lower: Cascade 85% · Absence 68% · Tracking 77% · Deletion 91% · Aggregation 1% · Exact Recall 100%. That is 72% overall against 92% with the descent, across the 694 questions, where raw memory scores 35%. The index is not an implementation detail: it is what takes Aggregation from 1% to 100%, because the hub already lists the three entities of the question with their value in force and one read answers all of them.
The Absence ceiling is an ambiguity of the benchmark itself: it writes those cases exactly like the Cascade ones, and no system can tell them apart reading only the conversation. Cascade and Deletion, by contrast, land exactly on their detection ceiling: with the descent, picking the wrong page stops costing answers and starts costing only tokens, because the index already states the status of every entity.
The two limits that remain, stated: Tracking stays at 77% for a lexical reason, not a representation one: the question uses the plural and the entity is singular, so the retriever scores zero and the page never opens. With plural normalisation it goes to 98% and the total reaches 95%, the exact detection ceiling. The second limit is scale: in a large brain the descent has to be able to open more than one hub. With the most pessimistic partition and a single hub, Aggregation falls back to 0%; opening 3 hubs, the total lands at 89%.
COMPARISON
Against what the market publishes.
The same categories, next to the only other numbers that exist in public. Read the conditions column before you read the figures.
| System | Cascade | Absence | Deletion | Condition |
|---|---|---|---|---|
| LAN Brain (measured by us) | 100% | 74% | 99% | 100 cases, nofiller, descent with an index, deterministic reader, public harness |
| Sentra (self-reported) | 40% | 43% | 50% | no methodology, harness or config disclosed · sentra.app, consulted 2026-08-23 |
| Field average (MEME paper) | 3% | 1% | n/p | the paper's default config (filler noise, LLM judge) |
Read this table carefully, on purpose. The rows are NOT directly comparable to each other: each one ran under different conditions, and Sentra's have no published methodology (their figures come from their own site, benchmarks section, consulted 2026-08-23). Ours is the only row with a reproducible harness and a declared config. The official MEME paper is under review and has not published its per-system table; when it does, or by running its official harness with API keys, the clean head to head comes out. Until then we prefer this honesty to a rigged chart.
EXPERIMENT 2 · COMPANY DATA
The same model, with and without LAN Brain.
20 real questions about LAN (decisions, dates, technical facts, and facts that changed over time), each with a verifiable answer in the brain. The same model answered twice: on its general knowledge alone, and connected to the company brain over MCP. The only variable is access to the context.
Model aloneModel + LAN Brain
- Accuracycorrect answers
- With a verifiable citationthe path of the page that backs the answer
- “I do not know”abstention: without the brain the model does not lie, it goes quiet
Internal run · N=20 · one model · rubric applied by the same system with mechanical criteria, not an independent judge · tokens estimated from character counts
| Metric | Model alone | Model + LAN Brain | Honest reading |
|---|---|---|---|
| Accuracy | 10% | 70% | 7x more correct answers |
| Hallucination | 0% | 5% | 1 case out of 20, caused by an out of date brain page, already corrected |
| Output quality (0-3) | 1.10 | 2.25 | correct, specific and cited versus “I do not have that information” |
| Speed (median) | 3.5 s | 7.5 s | with the brain it is 2x slower, but only 1.2 s of those 7.5 s is our server: the rest comes from the AI provider reading the sources |
| Total tokens | 1.5k | 17k | 11x more raw consumption |
| Tokens per correct answer | 762 | 1,213 | 1.6x the cost per useful answer, with 7x more useful answers |
- 1.2 sLAN server, measured in production
- 6.3 sprovider model inference plus network, outside our control
Our share of the extra time is 1.2 s: the other 6.3 s come from the AI provider reading the sources. Server time is measured in production, call by call; the rest is obtained by subtraction and is a decomposition, not a measurement, because provider inference and client network never reach our logs.
Efficiency is measured per useful answer, not per raw consumption: the model alone “spends little” because it barely answers at all (90% abstention). And the one invention in the brain arm was not invented by the model: it cited a brain page that had gone stale. That is exactly the problem change detection and re-verification of volatile facts attack, and the page was corrected the same day.
The honest claim: with LAN Brain the same model goes from an assistant that stays quiet (90% “I do not know”) to one that answers well and shows where it got it (70% correct, 80% cited), at 1.6x the cost per useful answer and double the latency, of which 1.2 s is our server. That is the trade, and we show all of it.
METHOD AND LIMITATIONS
What sits behind every number.
MEME.
Real dataset (HuggingFace meme-benchmark/MEME, config nofiller), 100 episodes, 694 questions. One single BM25 retriever for both arms, deterministic scoring with no LLM judge, identical result under any seed. The LAN Brain pages are built by an ingest simulator that reads the sessions in chronological order and detects supersede, deletion and dependencies on its own; it never sees labels or gold answers. And they are read by descent, the way the product reads: an index or area hub that lists the entities with their value in force, and from there to the page. Long values in the hub are truncated at 120 characters, the way a real index would, so Exact Recall forces a descent to the page and the hub cannot become a dump of the whole state. Measured detection: Cascade 100%, deletions 100% recall and precision, conditionals 100% precision, Absence 74% (the rest is a design ambiguity of the benchmark, documented with a provably undecidable case).
Company (20 questions).
Ground truth extracted from the internal wiki. The no brain arm answered blind to the answers: the questions were assembled by a separate process and the answering side only saw the prompt. Brain arm: a real read only MCP session, 33 calls. The latency attribution comes from the production percentiles of the server (1.65 calls per answer, p50 of 1,065 ms on search and 202 ms on a page read); what is not server we do not measure, and we state it as a subtraction.
Limitations, plainly.
One model; N=20, so 10% versus 70% is robust but the decimals are not; the quality rubric was applied by the same system with mechanical criteria, not an independent judge; tokens are estimated from character counts; the “model alone” arm is a controlled blindness simulation, not an API call with an empty context, and that version is the next step if this gets published.
Third parties.
No figure from another platform is presented as verified. Sentra publishes theirs with no methodology, and the MEME paper has not released its per system table. Figures that exist only as one vendor's citation of another system are not published here at all.
WHAT THIS MEANS FOR YOU
The same measurement, on your own data.
These runs used LAN's own brain and a public benchmark. The interesting version is the one run on your context, with your questions and your ground truth. That is part of the survey.
LAN Brain · internal benchmarks run 2026-08-23 · MEME: dataset HF meme-benchmark/MEME (CC-BY-4.0), harness SeokwonJung-Jay/MEME-public. The paper is under review and anonymous: there is no preprint to cite. Reproducible harness and scripts available on request. Sentra: figures self-reported on sentra.app, benchmarks section, consulted 2026-08-23. n/p = not published.