What LLMs actually do with a research wiki
A wiki-as-memory is only worth the trouble if the agent leans on it, and “leans on it” is easy to assume and hard to see. So we built a small tool that reconstructs every wiki touch from a session’s raw transcript, and ran it across the whole corpus of research and code projects. The behaviour is more specific, and more lopsided, than the pitch: the wiki is written far more than it is read, and the typed edges you author for the agent to follow are, in practice, written down rather than read back.
The instrument
An agent reaches a wiki three ways that a naive “count the tool calls” scan would split or miss: the Read tool, Bash grep/cat, and git (writing the wiki). The tool unifies all three, then asks, per user question, whether the answer was wiki-informed and by what route: a fresh fetch, reuse of a page an earlier turn already pulled into context, or the index injected at session start. Every number below is behaviour recovered from transcripts, not a projection.
Most questions never reach the wiki
Genuine questions are a small slice of what an agent does: 7.5% of turns (320 of 4,248); the rest are commands, edits, and running jobs. And of the content questions that dominate that slice (308 of them), most are answered without touching the wiki at all.
| of 308 content questions | share | |
|---|---|---|
| touch no wiki at all | 55% | 168 |
| reuse context already loaded | 39% | 121 |
| fetch a page fresh | 6% | 18 |
| the agent says it can’t answer | 0.2% | 2 of 1,244 |
The 55% is the number that surprises. It does not mean those questions are shallow. “Content” is only the classifier’s label for a substantive question, as opposed to a command or a relationship query; it says nothing about whether the wiki is needed. Read them and most are conversational follow-ups answered from the live dialogue already in context: a clarification, an “explain that”, a reference to a number produced two turns ago (“so the claims find the same entities?”, “for misses: markov ~4/12, can you explain?”). A content question is simply not the same thing as a wiki question.
Of the questions that do lean on the project, the dominant route is not a fresh fetch. It is reuse: a page an earlier turn already read, still sitting in context, informing a later answer. The SessionStart hook injects the wiki’s index at turn 0, so a map of every page is resident before the first question, and only 6% of content questions ever pull a page fresh. The wiki behaves like a resident reference the agent consults from memory, not a database it queries per question.
Written far more than read
The map below is the observed wiki: each node is a page, each arrow a move the agent actually made from one page to the next inside a single answer. The encoding carries what a link diagram cannot. A red arrow means the move landed on a write, an edit or a commit; a blue arrow means it landed on a read. A solid arrow means a frontmatter typed edge exists between the two pages; a dashed arrow means it does not. Green loops mark the agent re-reading a page it wrote earlier in the same turn: a solid loop is a genuine re-read, a dashed loop is a grep to check its own edit. Drag any node.
Two things read straight off the map. Red dominates. In llm-wiki-vision, 73% of moves land on a write; in yarp, 90%. The channel mix under the map says the same from the tool side: three-quarters or more of every wiki touch is a write or a commit. The wiki is touched mostly to maintain it, not to read it. Dotted dominates too. Solid frontmatter edges are a minority of moves in both wikis, so most navigation does not run along the links you authored. And traffic piles onto the two dark hubs, the injected index and log, which is the second finding.
Hub-and-search, not link-walking
The agent orbits the navigation pages. It reads a content page, returns to the index, and spokes back out, so the most central, load-bearing nodes in the observed wiki are exactly the ones you rarely think of as content. This is not link-walking at all. The injected index is a full catalog of every page, grep searches every page’s text, and a filename is just a token the agent can open, so every page is one hop from the hub. The agent does not walk the graph from a page to its neighbour; it teleports, jumping straight from the hub to whatever page the index names or grep finds. To the model the wiki is a flat, searchable set of pages, not a graph, and the authored edges between content pages are simply not on the path it takes.
The links are written, not read
That raises the sharpest question, and it is a correction to an earlier reading of this data. Are the typed edges you author, the related:/up: frontmatter, actually used to answer? Split every edge-follow on a question turn by what the agent did at the linked page, a read (an edge used to answer) or a write (the agent recording a relationship), and the picture inverts.
Across the whole corpus, on every question turn, an edge is followed to a read just 6 times. For the one question type edges exist to serve, relationship questions, an edge is read exactly once, against 22 times the agent followed an edge to write one. An independent typed edge is the path to an answer about 1 time in 320 questions. Reads reach their page by the index and by grep, essentially never by walking an authored edge.
So the honest conclusion is that the typed edge is an output of answering, not an input to it. The agent reads content, works out a relationship, and writes the edge to record it. It is worth being precise about the capability, because we tested it directly: on controlled fixtures where a page is reachable only through a frontmatter edge, its name absent from the prose, a blinded agent followed the typed edge and cited it 11 out of 11 times, and on a real densely-linked wiki it still picked the typed edge 6 out of 6 against 6 to 9 competing prose links. The model can answer from an edge. But when grep and the index are on the table, it does not; it greps. Capability is not usage. And a companion ablation found the typed edge adds zero answer value over the prose that already mirrors it (the wiki duplicates 89 to 99% of its typed edges into the body). So the leverage is not in authoring more categorical edges for the agent to read back. It is in capturing the structure the agent writes, and weighting it by use.
Read once, reuse many
The reuse finding is the same lesson from the read side. A page read early stays in the model’s window, literally cached, until a session compaction evicts it. The single sharpest number is the re-read rate, how often the agent re-fetches a page it has already seen rather than reusing it from context, and it ranges widely across the corpus: some sessions read a page once and reuse it for the rest of the session, others re-grep the same pages over and over. And the wiki’s fingerprint on answers is larger than a citation count shows: answers routinely quote a page’s body verbatim without naming it, and since the SessionStart hook injects page titles, not bodies, that quoted text can only have come from a page the agent actually read. The wiki is genuinely feeding answers; it is just doing it from resident context, not fresh lookups.
Is it “science reads, dev commits”? Not quite
It is tempting to read the two wikis as a type: the research project reads, the code project commits. The read/write balance really does range from mostly-reading to almost-entirely-writing across the corpus. But we cannot promote that to a law, because each wiki here is essentially one long session, and the balance reflects what that session was doing, not a fixed property of the project. yarp sits at 90% write because the captured session was a build-out; a later session that mostly queried the same wiki would read very differently. The read/write axis is real and useful, it separates a wiki being built from one being read, but it is a per-session signal on this data, not a per-project one.
What this is, and isn’t
This is a measure of behaviour, not value. It shows how the wiki is accessed, not whether it made an answer better, which is a counterfactual a transcript cannot show. But the behaviour is itself the finding, and it is specific: the agent leans on reused context, teleports through the index instead of walking the links, and writes the authored typed-edge graph far more than it reads it. The wiki works as memory the way the agent actually reads it: a searchable, hub-indexed pile of pages whose resident index answers most navigation, not a graph it traverses.
Honest caveats. This is access and behaviour, not answer-correctness; a transcript cannot show whether the wiki made an answer right, and “can’t answer” counts only explicit abstentions, so a confidently-wrong answer is invisible. Relationship questions are a tiny slice (11 in the corpus), so their numbers are a direction, not a rate, and the read-edge counts are small absolute numbers. The two maps are the captured sessions that touched each wiki, so the read/write balance is per-session, not per-wiki. The frontmatter ablation is a capability result on one model. Frontmatter is classified against current wiki text.