Agents are good writers and poor archivists

1. Summary

Each claim below is labelled. [measured] means our own experiments; the materials behind every number are available on request (Section 12). [literature] means published work we have read at the source.

1. [measured] Metadata, not page splitting, drives reading accuracy. In v1 (220 pages, about 6k–22k tokens, 82 questions), one flat document with a one-line metadata header per page scored 0.963–0.981, against 0.923–0.927 without the header and 0.970–0.974 for search-and-read tools. The flat document cost 2.6–4.5 times less per question.
2. [measured] The result held in a larger, harder world. In v2 (a generated consultancy with a ground-truth ledger, 108 pages, about 75k tokens as the model counts them, 219 questions in 11 categories), flat with today's metadata scored 0.90, flat without metadata 0.81 and tools 0.84. Flat with metadata beat bare flat clearly (22 questions against 4, p = 0.001). Its lead over tools (24 against 13) is not statistically significant (p = 0.10).
3. [measured] Richer metadata is a trade, not a gain. Adding validity dates, ranks and a provenance line per page raised "who decided or recorded this" questions from 0.60 to 0.95, but lowered event order (0.80 to 0.65) and multi-hop (0.85 to 0.75), for 0.87 overall. A count-and-group-by tool did not help (0.84 with and without it).
4. [measured] Agents write well, whatever the page size. Over 41 maintenance tasks, the same model working on many small pages or on a few large ones lost no facts, made no provenance errors and kept every replaced value as history. Task success was similar (0.81 granular, 0.83 large). Large pages cost more: 60% more unrelated content changed (19.5 against 12.2 per 100 tasks) and 80% more spend ($7.94 against $4.42).
5. [measured] Agents archive poorly. In both storage shapes the agent left 7 of 17 replaced values unmarked on an active page, and got 3 of 6 explicit "this replaces that" tasks fully right.
6. [measured] Nudges help, and overshoot. When every write answer pointed the agent at the pages it probably replaced, stale values fell from 7 to 4 and task success rose to 0.88. But the agent then marked whole pages as superseded when only one fact on them had changed, so current facts on those pages stopped reading as current (83 of 87 recoverable against 86 of 87).
7. [measured] A detector without a language model finds most replacements when the text says what changed. Our deterministic text matcher flagged 22 of 23 replaced pages in the v2 world with no false flags (precision 1.00, recall 0.96). When we removed the old values from the prose, as assistants often write, recall fell to 0.26.
8. [literature] Changing facts are where long-context models and memory systems fail. Models degrade well below their advertised window, fail first on counting, and give outdated answers when the context holds both old and new values [1], [2], [3], [4], [5], [6]. The memory systems that do better mostly attach validity and supersession metadata at write time [7], [8].

All of our numbers come from one model vendor (Claude Sonnet 5), one synthetic world per experiment, one run per condition and a few hundred questions or 41 tasks. Section 10 lists what that does and does not allow us to say.

2. The problem: a memory has to know what holds now

An assistant that works with a person over weeks needs to know what was decided, by whom, and whether it still holds. Chat products answered this with built-in memory. ChatGPT can "reference all your past conversations" [9]; one careful observer describes the result as a "dossier" injected into each new chat and asks for memory scoped per project [10]. Claude's app memory is scoped per project and can be read and edited [11]. Claude Code keeps an index file plus one file per memory, each with a type field and a timestamp "so the timestamp shows how current the fact is" [12]. Memory is moving from a hidden store to files that people and agents can both inspect.

Once memory is a set of documents, three questions follow. What shape should the documents take? Should the agent read all of them or search? And who keeps them in order as facts change?

The first two have received most of the attention. The third is the subject of this paper, because it is where the evidence points. A memory accumulates contradictions by design: the deploy window was Tuesday, then Thursday; the quote was 84,000 kronor, then 91,500. Each value was true when written. What the memory must add is the ordering: which value holds now, which held before, and when it changed.

The literature shows how much rides on that ordering:

• Outdated facts are the main failure in question answering over changing knowledge. FreshQA found GPT-4 at 28.6% strict accuracy without search, 12% on fast-changing facts, and 75.6% with search results in the prompt [5]. RealTime QA notes that models "tend to return outdated answers when retrieved documents do not provide sufficient information" [13]. LongMemEval, which tests knowledge updates, measured GPT-4o at 87.0% with only the relevant evidence against 60.6% with the full ~115k-token history [6]. On MemoryAgentBench's multi-hop fact-consolidation task, every method tested scored at most about 6% [14].
• Long context is not a substitute for ordering. RULER found only about half of 17 models holding their short-context performance at 32k tokens [1]. NoLiMa removed word overlap between question and answer: at 32k, 11 of 13 models fell below half their short-context score [2]. Chroma's context-rot study found that a single distractor hurts and four lead models to hallucinate from them [3]. A superseded value is the most convincing distractor there is: same subject, same wording, a different number.
• Aggregation fails first. Oolong asks models to classify and count across a long text; GPT-5, Claude Sonnet 4 and Gemini 2.5 Pro all scored below 50% at 128k [4]. "How many decisions are still active?" is an aggregation over the ordering.

The memory systems that beat plain long context do so mostly by recording the ordering when a memory is written. Zep gives every graph edge a validity interval and invalidates an edge when new information contradicts it [7]. A-MEM attaches keywords, tags and links to every note as it is stored [8]. Wikidata, the largest human-and-bot knowledge base, ranks every statement as preferred, normal or deprecated, keeps once-true values with start and end times, and by default returns only the best-ranked values [15], [16]. Where the whole history is short enough, plain long context often matches these systems on accuracy [17], [18], [19]; the question is what happens as it grows and changes.

Our experiments asked the two easier questions first (shape and reading strategy) and found that the third one (ordering) decided the answers.

3. What we tested

We ran four experiments between 1 and 4 October 2026, all with Claude Sonnet 5 (extended thinking off) on Wunel's own tools and core library, against a local database.

Token counts for v2 are the input the model actually processed per question (Claude's tokenizer), which runs higher than the o200k estimates in our fixtures. The total spend, pilots included, was $55.50. We also ran an offline check of a text-matching detector, which costs nothing to run.

4. Reading, v1: the metadata line did the work

4.1 Setup

Corpus. Three synthetic workspaces from our search test fixtures: a team workspace (85 pages), an assistant-memory workspace (64 pages, with 5 superseded facts and 1 archived decision) and a story archive (71 pages).

Questions. 82 questions with gold answers written from the source pages: 33 lookup, 14 multi-hop, 15 aggregate, 12 current-versus-superseded and 8 unanswerable; 56 in English and 26 in Swedish.

Conditions. Same model, same instruction ("answer from the workspace only, briefly; prefer current information; say UNKNOWN if it isn't there"):

• flat: all pages concatenated under their titles, frontmatter removed. This is what a person gets by pasting their notes into one page.
• flat + metadata: the same, plus one line per page with slug, type, status, supersedes, a derived superseded_by, and last_verified.
• tools: no document; the product's ranked search, a frontmatter filter, an index, file listing and read tools, capped at 8 model calls.

Each question was asked against its home workspace (78 questions) and against all 220 pages (82). Answers were graded by exact match and by an LLM judge with a strict rubric, in which an outdated value given as current is wrong. The decision rule, fixed before the run: flat counts as "as effective" if it is within 3 points of tools overall and no category is more than 10 points worse.

4.2 Results

The bare flat page lost about 20 points on aggregation at both sizes (0.733 and 0.667 against 0.9 and 0.867 with metadata) and 20 points on current-versus-superseded at workspace size (0.8 against 1.0). Lookup and multi-hop were perfect in every flat condition.

Flat with metadata met the decision rule: +0.7 points against tools at workspace size and −0.7 at the larger size. The only thing it had that bare flat lacked was a line per page saying what kind of page it is, whether it is current, and what replaced it. Asked on which day production deploys happen, bare flat gave the old value; with the metadata line the model gave the current one. Asked how many decisions were still active, bare flat said the workspace "does not clearly mark" which notes are superseded; with the metadata line the model counted the pages with type: decision and status: active and gave the right answer.

v1 had obvious weaknesses: a small hand-built corpus with a handful of superseded facts, easy lengths, gold answers read back from the same pages, and a judge from the same model family. v2 was built to address them.

5. Reading, v2: the same answer in a harder world

5.1 The memory world

v2 replaces the hand-made corpus with a generated world whose ground truth is a ledger: people, entities, facts with validity intervals and status, and events that each render to a page. Gold answers are computed from the ledger at a cutoff date, never read back from page text, so the same question can have a different correct answer at different sizes.

The world, "renovation", is a six-person architecture and renovation consultancy in Stockholm over January–March 2026, with two client projects whose vocabulary overlaps (a kitchen and bathroom renovation, and an office refit with a kitchenette), a role change in February, recurring meeting series, decisions, emails, quotes, incidents and drafts, and near-miss distractor pages. The ledger has 106 events, 122 facts (20 superseded, 4 retracted) and 8 people. The pages are in Swedish and English, as a Swedish firm with an English-speaking client would write them.

Two sizes were run: S (January, 31–33 pages) and M (January–March, 106–108 pages). The same content was rendered in three forms:

• raw: titles and prose, no frontmatter;
• v1: today's Wunel defaults: type, status, supersedes / superseded_by, sources, purpose, last_verified, an AGENTS.md and an index page;
• v2: v1 plus valid_from / valid_until, a rank (preferred, normal, deprecated), typed relations, and a provenance line per page (written_by, on_behalf_of, verified_by).

Questions. The question bank has 309 questions in 11 categories (lookup, multi-hop, exact counts, lists, current value, value as of a past date, event order, provenance, abstain, false premise, and low word overlap). To stay within budget the run asked about 20 per category: 219 at M and 127 at S, where some categories have few or no valid questions.

Conditions. Flat raw, flat v1 and flat v2 (each a single cached document); tools over v1 pages (the product's search and read tools, at most 8 calls); tools plus a count-and- group-by tool; and an oracle that sees only the pages that support the answer.

Grading. Programmatic first (numbers, dates, names, lists, UNKNOWN, false-premise corrections), with an LLM judge (Claude Haiku 4.5) for the 14% of answers that need one, and the order of reference and candidate randomised, because LLM judges have been shown to flip verdicts when answer order is swapped [20]. A pilot run on 3 October led to a review of every failed answer: 25 of 37 failures were errors in our gold answers or questions, 5 in the oracle's page selection and 1 in the grader. We fixed those and re-ran.

5.2 Results

Strict accuracy, 95% Wilson intervals:

Because every condition answered the same questions, the fairer comparison is per question. Counting the questions one condition got right and the other wrong:

Strict accuracy by category at M (n = 20 per category except lookup 27, lists 19, abstain 13):

5.3 What it shows

Metadata still carries the gain, now with a clear margin. At M, flat v1 beat bare flat by 9 points, and the per-question comparison leaves little doubt (22 against 4). The gain sat where v1 predicted and in one place v1 could not test: event order (0.50 to 0.80; 6 against 0, p = 0.03), counts (0.70 to 0.80) and abstention (0.85 to 1.00). With metadata the model never gave a superseded value as current; bare flat did so 3 times.

Flat with metadata did at least as well as tools, at lower cost and a third of the time. Flat v1 led tools by 5 points (0.90 against 0.84), though with 24 questions against 13 the lead could be chance. Tools gave a superseded value 5 times, flat v1 never. Tools were weakest on abstention (0.69): a search that returns near-misses invites an answer. On cost the gap has narrowed since v1: with the document cached, flat v1 cost $0.0169 per question against $0.0201 for tools, while reading 75k tokens per question against 12k. Tools took 4.9 seconds and about three model calls; flat took 1.9 seconds and one.

In v1 flat was 2.6–4.5 times cheaper; at 75k tokens it is 1.2 times cheaper. The point where reading everything costs more than searching is not far above this size for this model and price list, and caching is what keeps flat reading competitive at all.

Richer metadata is a trade. Adding validity dates, ranks and a provenance line per page (v2 DNA) helped at S (0.90 against 0.88) and hurt at M (0.87 against 0.90); neither difference is significant. Per category, the picture is sharper. Provenance questions ("who recorded the revised quote?") rose from 0.60 to 0.95 (19 of 20 against 12; 8 against 1, p = 0.04): when the page says who wrote it, the model reads it. But event order fell from 0.80 to 0.65, multi-hop from 0.85 to 0.75, and abstention from 1.00 to 0.85, and v2 gave a superseded value twice where v1 gave none. The v2 document is 11k tokens longer, and much of that is dates: a valid_from, valid_until and a provenance date on every page compete with the dates in the text when the question is "what happened after X?". Our reading is that each metadata field helps the questions it answers and costs a little on every other question, through length and distraction. Fields should earn their place.

A count tool did not help. Counting is where v1 and the literature both found models weakest [4], [21], so we gave the tools condition a frontmatter count with group-by. Overall accuracy did not move (0.84 with and without; 9 against 10 questions), exact counts went from 0.75 to 0.70, and the condition gave the most superseded values of any (7). The model used the tool, but counting pages by type and status is only right if the status is right, and the count tool trusts it completely. A count over a stale ordering is a precise wrong number.

The oracle is not a ceiling. The oracle condition, which sees only the pages that support the answer, is meant to separate reasoning errors from errors caused by length. It scored below flat v1 at both sizes (0.80 and 0.86), mostly because it said UNKNOWN when an answer existed (15 times at each size): shown three pages, a careful model cannot know that nothing else bears on the question, even when told so. Counting and "who recorded this" were its weakest categories at S (0.39 and 0.29). We report it, but we do not use it to claim that length cost the flat conditions anything at these sizes; at 75k tokens we see no sign that it did.

What v2 did not test. We planned a length ladder up to 1M tokens and a run with deliberately wrong metadata. Neither was built or run; the largest size we measured is about 75k tokens of model input. Whether flat reading holds at 200k or 800k tokens, where the literature finds most models degrading, is still open.

6. Writing, v2: agents are good writers

Reading is half of what a memory does. The other half is keeping it up to date. We expected storage shape to matter most here: an agent maintaining one large page that holds everything has more to drop, and every edit touches the same page.

6.1 Setup

The write test starts from the world at the end of January (size S) and gives the agent 41 maintenance tasks drawn from the ledger, one per event from 2 February to 6 March 2026, in the words a person would use: "Today is 2 February. We got an email from Erik Holm: demolition now starts on 9 February instead of 2 February; the container moves to the same day. Update the workspace." The tasks cover recording a new value that replaces an old one (6), new facts (12), a meeting's decisions (10), corrections (3), tasks and checklists (5), a rename and a move, and three distractions that need no change. Each task is a fresh conversation, with up to 12 model calls.

The agent gets the real Wunel MCP tools and server instructions, over the real api running in process, so block-level merge, edit guards and provenance behave as in production. It is run twice from the same content:

• granular: the v1 pages, one per meeting, decision, email or quote (33 pages);
• large: the same content consolidated into 8 large pages, one per project or area, a log and a task list.

After every task and at the end, the harness checks against the ledger whether each fact is still recoverable, whether each current fact reads as current, whether each replaced value is kept but marked as replaced, whether content the task did not target changed, and whether every agent block is attributed to the agent acting on behalf of the person. A replaced value counts as stale if it still appears on an active page without a mark.

6.2 Results

6.3 What it shows

The agent is a good writer. In 82 task runs it lost no fact, deleted no history, wiped no checklist and wrote nothing under the wrong name. Every replaced value survived somewhere as history. Wunel's edit guards (a shrinking list is refused, a write that deletes blocks says which) fired twice and the agent recovered.

Page size mattered less than we expected, and where it mattered it was cost and collateral. We had hypothesised that large pages would lose content. They did not. What large pages did was change more of what the task did not ask about (8 unrelated blocks against 5 over 41 tasks; 11 blocks deleted by whole-page writes against 2) and cost 80% more, because every read and write moves a larger page. The absolute counts are small, and a single run cannot separate a 60% difference from noise. We would not build a product argument on the write-side accuracy of small pages. The case for small pages rests on cost, on legible history (a change to a small page is a sentence, a change to a large one is a diff), on sharing and permissions, and on the archiving problem below.

Neither shape solved archiving. In both runs the agent left 7 of the 17 replaced values looking current, and got 3 of the 6 explicit replacement tasks fully right. The stale values were ordinary: the weekly meeting moved from Monday to Tuesday, the kitchen countertop changed from solid oak to composite stone, a container date moved, a furniture quote was revised. In each case the agent recorded the new value correctly, often on a new page with a good title, and left the old page saying the old thing with nothing to mark it as replaced.

This is the pattern behind the title. Writing a new fact is a local act: the agent has the new information in front of it and a page to put it on. Archiving is a non-local act: the agent has to find every older page that says something the new fact contradicts, decide whether it is contradicted or merely related, and mark it, in a workspace it sees through search. The new page is correct; the workspace as a whole is now wrong. A reader, human or model, who finds the old page first will get the old answer.

The read side shows the cost. After the 41 tasks, flat v1 questions over the resulting workspace scored 0.71 in both shapes, against 0.90 on the world as generated, with the losses concentrated in counts and lists (0.48–0.65). Part of that gap is our questions (the post-write question set is small and count-heavy: 94 questions, 27 of them counts), but counts are exactly what a stale ordering breaks.

7. Nudges help, and overshoot

If the agent fails to find what it replaced, the system can find it and say so. On 4 October we added a write-time pointer to Wunel. When an assistant writes a page, the server looks for earlier pages that the new one probably updates (Section 8 describes how) and, if it finds any, adds a short note to the write answer:

This page looks like it updates 'Inköpsgenomgång 1' (file_id 4f…: 84 000 kr → 91 500 kr). If it replaces it, set supersedes: Inköpsgenomgång 1 on this page and status: superseded and superseded_by: Reviderad VVS-offert on that one; if not, ignore this.

We re-ran the granular write test with the pointer in place, same tasks, same model:

The pointer did what it was for: stale values fell from 7 to 4 and success rose by 7 points. It also showed the next failure. Three current facts about the office project (its address, its budget, its floor area) stopped reading as current. The cause was one task: the client's steering group raised the number of workstations from 28 to 32 and moved the move-in date. The agent recorded the decisions on a new page and marked the original client brief as superseded, because two facts on it had changed. The brief also held the address, the budget and the area, which had not changed, and now sat on a page marked as replaced.

Unrelated changes also rose (11 blocks against 5), as the agent went back to older pages to mark them. With one run per condition we cannot say how much of that is the pointer and how much is variation between runs.

So the agent overshoots once told to archive. Its unit of archiving is the page, while the unit of truth is the fact. A page that holds ten facts, one of which changed, is not superseded; one sentence on it is. Wunel's convention lets an agent mark a section or a block as replaced, but the agent reached for the page-level field the pointer named. The lesson for the product is that the pointer must name the fact, not just the page, and that marking a whole page as replaced should be the agent's claim that nothing on it still holds.

8. Finding replacements without a language model

The pointer in Section 7 depends on the server knowing which earlier pages a new page probably updates. Wunel promises not to run AI models on the content of people's pages, so the detector is deterministic text matching.

How it decides. Each earlier page gets a score from a handful of signals; a candidate needs 3.0. The strongest is a stated change: the new page says "91 500 kr, tidigare 84 000 kr" ("previously"), "2,32 m, inte 2,40 m" ("not"), "från 4 mars till 11 mars", "kompositsten i stället för massiv ek" ("instead of"), and an earlier page still gives the old value in a sentence about the same thing (a shared word such as "offert" or a supplier's name). Weaker signals are a conflicting value (same unit and label, the same named thing, a different number), the same type and title once dates and version markers are stripped, the same subject key, and a link with words like "rättelse" or "replaces". It understands amounts, measures, dates, times and counts in Swedish and English.

Guards that keep it quiet. Most of the work went into not flagging. Recurring pages such as meeting minutes never match on title alone, because historic minutes are not stale. Pages naming different months never conflict, because January's invoicing is not February's. A change that an older page already describes is not news. Pages about different named clients or suppliers do not match. A false pointer costs the writer a call and costs Wunel trust, so the detector errs towards silence.

Offline check. We replay the v2 world: render every page with the supersession fields removed, write the pages in date order, and record what the detector flags at each write. The gold answer is the ledger: for each fact with a successor, the page setting the new value should flag the page setting the old one.

Read these numbers with care. The world's prose almost always states the change with the old value, and the detector was tuned on this world, so 1.00 and 0.96 are an upper bound. The second row is the honest floor: when the new page states only the new value, which is how assistants often write ("demolition starts 9 February"), the detector finds conflicting amounts, measures and counts, and misses moved dates, changed roles, materials and weekdays. One write in a workspace of about a thousand pages takes about 5 ms.

This bounds what a server can do alone. Text matching can find "84 000 kr then 91 500 kr" reliably. It cannot know that "the steering group moved move-in to 11 May" replaces "move-in on 4 May" unless the text says so, and it cannot know that the brief's address still holds. Those judgements belong to something that understands the text: the person's own assistant, or the person.

9. Why ordering has to be owned

Three observations from the experiments fit together.

First, the reading results say that the ordering is what makes a memory answerable. The metadata line that carried v1 and v2 is mostly ordering: status, supersedes, superseded_by, last_verified. With it, the model gave no superseded value as current in either experiment; without it, it did. Tools and a count tool did not substitute for it, because search and counting both inherit whatever ordering the pages carry.

Second, the writing results say that agents do not maintain the ordering unprompted, and that page size does not change this. An agent that writes a correct new page has done the part of the job it can see.

Third, the nudge results say that telling the agent helps but moves the error. Asked to archive, the agent archives at the granularity of the field it is pointed at.

This is an old problem in a new place. Structured metadata on the web and in public-sector catalogues failed where people had to write it by hand and spread where it was light and rewarded [22], [23], [24]. Doctorow's "Metacrap" listed "people are lazy" and "people lie" among seven obstacles, and singled out metadata observed by the system as "far more reliable than the stuff that human beings create" [22]. Sweden's mandated open-data catalogues show the same friction: even with an EU directive behind it, a review of 40 agencies found that they "seldom" provide structured metadata [23], and the people who create the data are seldom the ones who publish it [25].

Language models change half of this. They write type, date, status and links as a by-product of writing the page, at almost no cost; in our write test the agent set last_verified in 34 of 41 tasks without the task asking for it. A-MEM reports about 1,200 tokens per memory operation to attach keywords, tags and links [8]. Karpathy's "LLM Wiki" pattern has the model maintain the wiki and adds a "lint" pass for "contradictions, stale claims, orphaned pages" [26]. What models do not change is the other half: the relation between a new page and every older one. That relation is a property of the whole memory, not of the page being written, and it is the part that every assistant, from every vendor, writing into the same workspace, would have to get right independently.

A memory that several people and several assistants share cannot leave the ordering to whichever writer happens to remember it. The store has to own it: detect what it can, ask the writer about the rest, and give the person the final say. Zep does the first in a graph by invalidating edges on write [7]; Wikidata does the third with ranks that people and bots maintain [15], [27]. A document memory needs all three.

10. Limitations and counter-evidence

One model vendor. Every answer and every write in this paper came from Claude Sonnet 5, and the v2 judge was Claude Haiku 4.5. Claude models tend to abstain rather than guess [3], which flatters the abstain and false-premise categories and penalises wrongly saying UNKNOWN. Other vendors' models may write, read and archive differently.

One run per condition. Each v2 cell is a single run. The intervals in Section 5.2 are over questions, not over runs, and the write-test differences (5 against 8 unrelated blocks, 7 against 4 stale values) are counts from one sequence of 41 tasks each. We report them as what happened, not as effect sizes.

Synthetic world, partly self-graded. The memory world was generated with the help of a language model and checked for consistency between ledger and prose. It is cleaner than real notes: it almost always states changes with the old value, which favours the detector, and it has few near-duplicates. Gold answers come from the ledger, not from the pages, and most grading is programmatic, but 14% of answers went to an LLM judge, and our pilot review found that most early "failures" were our own gold errors. Others may be left. The write test's "stale" check is strict: it counts an old value on an active page even when the new page mentions it as background ("Katarina originally wanted solid oak"), so some stale counts may overstate the problem.

Small n and moderate length. Some categories have 13–20 questions, so one answer moves a category by 5–8 points. The largest context we measured was about 75k tokens, far below the 1M windows now offered and below the lengths where the literature finds the steepest decline. The write test ran 41 tasks over five weeks of a small firm's life; a real memory runs for years.

The tools condition is a floor. It depends on our search, which in v1 had recall@5 of 0.53–0.57 for aggregation questions. A better retriever would narrow the gap to flat reading.

Retrieval can beat long context. Zep reports 71.2% on LongMemEval with about 1.6k tokens of retrieved memory against 60.2% for GPT-4o with the full ~115k context [7]. OP-RAG found 48k tokens of ordered retrieval beating 117k of full context [28]. Chroma's 300-token focused prompt beat the 113k one [3]. LaRA concludes there is "no silver bullet": the winner depends on model, length, task and chunking [29]. Our claim is bounded to the sizes we ran.

Memory benchmarks are disputed. Mem0's paper reported full context ahead of Mem0 and Zep on LoCoMo [17]; Zep replied that Mem0 had configured Zep wrongly and gave a corrected figure [30]; Letta reported 74.0% on LoCoMo for an agent with only file tools [31]. Numbers in this field move with configuration; ours will too.

What we did not test. Concurrent human edits during agent writes (the harness supports it; we did not run it), wrong metadata, the length ladder above 75k tokens, a second world in another domain, and the human review step proposed in Section 11.

Conflict of interest. We build a product whose design these results inform, and the detector was built by us and tuned on our own world. We fixed v1's decision rule before running it, we report the categories where our preferred conditions did worse, and the materials behind every number are available on request (Section 12). Until they are public, readers have to take our grading on trust, which is a real limitation.

11. Design implications

The findings point to one design principle for shared document memory: the memory system owns the ordering of what is current. Assistants write the pages; the system decides how the question "is this still true?" gets asked and answered. In Wunel this is three layers, cheapest first. Wunel runs no AI models on page content; the language understanding in layers 2 and 3 comes from the person's own assistant and from the person.

Layer 1: the server detects, without a language model. On every assistant write, the deterministic detector from Section 8 looks for earlier pages that the new one probably updates. Because it cannot read meaning, it is tuned for precision and stays silent when unsure. The same detector feeds a likely_superseded list in the workspace health check, so pairs that slip through at write time are still found later. Wunel: in production since 4 October 2026.

Layer 2: nudge the writer at write time. The detector's findings go to the assistant that just wrote, in the answer to its write, while it still has the context. The assistant, which can read meaning, decides whether the older page is replaced. Section 7 shows this cuts stale values and that it needs care: the note should name the fact that changed and the sentence it sits in, and should say that marking a whole page as replaced means that nothing on it still holds. Wunel: page-level notes ship today; fact-level wording is the next change, to be measured with the same write test.

Layer 3: the person says yes or no. Some orderings neither text matching nor an assistant can settle: whether a fact checked in January still holds in April, or whether the client's address changed when nobody wrote it down. These go to the person as a bounded question. Wunel: a "Still true?" review, in testing at the time of writing, asks the person about at most ten pages nobody has confirmed in 90 days. Confirming sets last_verified to today; otherwise the person corrects the page or asks their assistant to update it. A "May be out of date" list shows the detector's pairs with the newer page next to each.

The rest of the findings shape the read and write paths around these layers:

1. Return the best rank by default. In a flat view or a search result, a replaced page should be one line ("replaced by X on D"), not its full text, unless history is asked for. This is Wikidata's "truthy" default [15], and Chroma's distractor result argues for it [3]. Wunel: search ranks superseded and archived pages after current ones; in our search evaluation no superseded page ranked above its replacement in 9 test cases, against 4 of 9 before.
2. Offer a "read the whole memory" operation, sized by the caller. Flat reading with a metadata line per page was as accurate as tools in both experiments, at a third of the latency. Only the client knows its context window and price, so the answer should state the memory's size first and let the agent read up to its own budget and page through the rest. Wunel: planned for the next reviewed tool release.
3. Make metadata fields earn their place. Status and supersession links helped every question type. Per-page provenance helped "who decided" questions and cost a little elsewhere; validity dates did not help at our sizes. Include provenance in a read when the question needs it, not on every page by default.
4. Supersede at the level of the fact. A page holding several facts should be able to say that one of them is replaced. Wunel: a section or block can be marked as replaced today; the nudge should point there first.
5. Do not count on a count tool. A count over a stale ordering is precise and wrong. Fix the ordering first; a count tool is a convenience after that.
6. Keep pages small, for cost and legibility rather than safety. Large pages did not lose facts in our test, but they cost 80% more to maintain and changed more of what the agent was not asked to touch. Small pages also make history legible and sharing possible.
7. Keep the edit guards. Refusing an edit that shrinks a list, naming the blocks a whole-page write deleted, and attributing every block to "assistant, on behalf of person" cost little and held: no facts lost and no provenance errors across 123 task runs.
8. Lint softly. Save always; flag what is missing or probably stale; let an assistant or the person fix it. A memory that refuses an incomplete page loses the page [32], [33].

12. Reproducibility

The memory world (ledger, page renderer, questions and gold functions), the read and write harnesses, the detector's offline check and the raw outputs of every run in this paper are not public yet. They are available on request, and we may release them later. Until then, nobody outside Wunel can rerun the experiments or check our grading, and the numbers here should be read with that in mind. A rerun needs an API key and costs roughly what Section 3 lists.

13. About Wunel

Wunel is a collaborative markdown workspace, built by Upnorth AI AB in Sweden, in which people and AI assistants write the same pages. Each block records who wrote it and on whose behalf ("an assistant, on behalf of a named person"), assistants connect over MCP, and people can review assistant edits block by block. Wunel runs no AI models on page content: the detector in Section 8 is text matching, and every judgement about meaning is made by the person's own assistant or by the person. The experiments in this paper ran against Wunel's own search, read and write tools.

References

All sources were opened and read at the source on 1 October 2026.

1. Hsieh et al., "RULER: What's the Real Context Size of Your Long-Context Language Models?", Apr 2024. https://arxiv.org/abs/2404.06654
2. Modarressi et al., "NoLiMa", ICML 2025. https://arxiv.org/abs/2502.05167
3. Hong, Troynikov & Huber, "Context Rot", Chroma, 14 Jul 2025. https://www.trychroma.com/research/context-rot
4. Bertsch et al., "Oolong", Nov 2025. https://arxiv.org/abs/2511.02817
5. Vu et al., "FreshLLMs" (FreshQA), Oct 2023. https://arxiv.org/abs/2310.03214
6. Wu et al., "LongMemEval", ICLR 2025. https://arxiv.org/abs/2410.10813
7. Rasmussen et al., "Zep: A Temporal Knowledge Graph Architecture for Agent Memory", Jan 2025. https://arxiv.org/abs/2501.13956
8. Xu et al., "A-MEM: Agentic Memory for LLM Agents", NeurIPS 2025. https://arxiv.org/abs/2502.12110
9. Altman, via OpenAI Developer Community, "ChatGPT can now reference all past conversations", 10 Apr 2025. https://community.openai.com/t/chatgpt-can-now-reference-all-past-conversations-april-10-2025/1229453
10. Willison, "ChatGPT's new memory", 21 May 2025. https://simonwillison.net/2025/May/21/chatgpt-new-memory/
11. Anthropic, "Memory" (Claude app), 11 Sep 2025. https://claude.com/blog/memory
12. Anthropic, Claude Code documentation, "Memory". https://code.claude.com/docs/en/memory
13. Kasai et al., "RealTime QA", 2022/2024. https://arxiv.org/abs/2207.13332
14. Hu, Wang & McAuley, "Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions" (MemoryAgentBench), Jul 2025. https://arxiv.org/abs/2507.05257
15. Wikidata, Help:Ranking. https://www.wikidata.org/wiki/Help:Ranking
16. Wikidata, Help:Qualifiers. https://www.wikidata.org/wiki/Help:Qualifiers
17. Chhikara et al., "Mem0", 28 Apr 2025. https://arxiv.org/abs/2504.19413
18. Li et al., "Retrieval Augmented Generation or Long-Context LLMs?" (Self-Route), EMNLP 2024. https://arxiv.org/abs/2407.16833
19. Pollertlam & Kornsuwannawit, Mar 2026. https://arxiv.org/abs/2603.04814
20. Han et al., "RAG vs. GraphRAG: A Systematic Evaluation and Key Insights", Feb 2025. https://arxiv.org/abs/2502.11371
21. Zhang, Kraska & Khattab, "Recursive Language Models", Dec 2025. https://arxiv.org/abs/2512.24601
22. Doctorow, "Metacrap", v1.3, 26 Aug 2001. https://chnm.gmu.edu/digitalhistory/links/pdf/preserving/8_17.pdf
23. Statskontoret, rapport 2018:6, "Hinder för att använda myndigheternas öppna data". https://www.statskontoret.se/siteassets/rapporter-pdf/2018/2018-6.pdf
24. Web Data Commons, structured data statistics, October 2024 crawl. https://webdatacommons.org/structureddata/2024-12/stats/stats.html
25. Reinsfelt, master's thesis, Mid Sweden University, 2022. https://www.diva-portal.org/smash/get/diva2:1677570/FULLTEXT01.pdf
26. Karpathy, "LLM Wiki" (gist), 4 Apr 2026. https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f
27. Piscopo, 2018. https://arxiv.org/abs/1810.00931
28. Yu et al., "In Defense of RAG in the Era of Long-Context Language Models", Sep 2024. https://arxiv.org/abs/2409.01666
29. Li et al., "LaRA: Benchmarking Retrieval-Augmented Generation and Long-Context LLMs", Feb 2025. https://arxiv.org/abs/2502.09977
30. Zep, "Lies, Damn Lies, & Statistics: Is Mem0 Really SOTA in Agent Memory?", 6 May 2025 (updated 2026). https://blog.getzep.com/lies-damn-lies-statistics-is-mem0-really-sota-in-agent-memory/
31. Letta, "Benchmarking AI Agent Memory: Is a Filesystem All You Need?", 12 Aug 2025. https://www.letta.com/blog/benchmarking-ai-agent-memory
32. W3C, SHACL, Recommendation, 20 Jul 2017. https://www.w3.org/TR/shacl/
33. EntryScape documentation, Catalog / Datasets. https://docs.entryscape.com/en/catalog/datasets/