Knowledge: Extract, Retrieve, Synthesize

From layout-aware extraction to durable, wiki-grounded research memory

Priscila Moreira
Charles Vardeman
James Sweet
Chris Sweet

July 9, 2026

From PDF to structured elements

Stage 1 · Extraction · paper structure, live

The document graph, live

Stage 2 · Ingestion · real output from the extraction pipeline

The extraction pipeline turns a source document into a typed RDF graph: the document at the center, section chunks around it, and the concepts each chunk mentions fanning out. Every concept carries two @type slots — one canonical (SPARQL-queryable across the whole corpus) and one domain-specific (grown from the source, no pre-declaration). The graph on the left is a real slice from a real briefing; the JSON on the right is the actual output.

{
 "@id": "did:doc:a5bf...",
 "@type": "prov:Entity",
 "schema:name": "CLAIMS-BRIEF",
 "schema:hasPart": [{
  "@id": "did:heading:9da2...",
  "@type": [
   "schema:CreativeWork",
   "ex:heading_chunk"],
  "ex:heading": "Design principles",
  "skos:member": [{
   "@id": "concept:late_binding",
   "@type": [
    "skos:Concept",
    "ex:design_principle"],
   "skos:prefLabel":
    "Late binding everywhere"
  }]
 }]
}

Two @type slots: canonical → SPARQL, ex: → corpus-grown.

Two kinds of “hop”

Terminology · reasoning-hops vs retrieval-hops

Multi-hop in the literature conflates two different concepts.

Reasoning hop

A property of the question. One step in an inference chain, fixed by dataset annotation.

MuSiQue’s “2-to-4 hop questions” are compositions of connected single-hop questions, each depending on the previous answer.

Trivedi et al., TACL 2022

Retrieval hop

A property of the system’s execution. One retrieval iteration, or one edge traversed in a graph, at run time.

What our random walk does. Walk length is a retrieval-hop count.

HopRAG · IRCoT

These don’t need to align. Even single-reasoning-hop questions need multiple retrieval hops to build the context envelope (layout, related concepts, cross-document relationships) around each retrieved chunk.

Dense RAG breaks on the bridges.

The retrieval problem

Dense RAG: embed the question, cosine against chunk embeddings, take top-K, hand to the LLM. One-shot similarity.

  • Works for hop 1 — the passage that lexically matches the question.
  • Breaks on multi-hop — bridge and answer chunks share no vocabulary with the question.
  • Breaks at scale — with a real corpus of distractors, similar-but-wrong passages drown the true evidence.

Multi-hop retrieval is a graph traversal problem in disguise.

Measured · 21,100-chunk corpus · MuSiQue TEST

Rank 1
The direct-match gold chunk. Stays at rank 1 at any corpus size.
~1,800
The 4-hop bridge chunk. Median rank of 21,100.
22%
all-gold@30 · vs 88% on a small distractor-free set.

Dense retrieval is a hop-1 tool. Multi-hop needs structure, not more similarity.

Markov chain: retrieval as a random walk

Live simulation · markov_embeddings_and_rag

The missing layer: durable synthesis

Transition · from retrieval to memory

Extraction gives you records. Analysis gives you answers. Neither gives you memory — knowledge that compounds across sessions and researchers.

Today that understanding lives in someone’s head, in stale notebooks, in README fragments. It walks out when the person leaves.

Where knowledge accumulates today

  • Presentation · papers, reports
  • Synthesis · [ MISSING ]
  • Analysis · notebooks, dashboards
  • Storage · PostgreSQL, PubMed, WoS

What a synthesis layer needs

  • Persistent · Markdown in git, grows monotonically
  • Human + agent readable · one substrate, no lock-in
  • Cited & traceable · every claim links to source
  • Queryable as a graph · typed edges → SPARQL KG

llm-wiki-memory-template fills this layer. The rest of the talk is what that looks like in practice.

Two ways in

The template’s two ingestion modes

Direct-ingest · schema-extraction into wiki pages

For a group’s own papers, a thesis lit review, a research-execution wiki.

Each source becomes a typed page (SourceSummary, LiteratureNote, MethodNote) with frontmatter edges (extends, supports, criticizes, up).

The KG is over pages — categorically different from the per-document parse graph on slides 2-3.

markov_embeddings_and_rag, AI-Sci-Disc-Mem, SavoieAgentKnowledge, web_forager — all direct-ingest.

Synthesis over a pipeline · above the document-parse graph

For large corpora where you first run extraction + retrieval — slides 2-5’s paper-explorer, document graph, 21,000-chunk MuSiQue.

The pipeline document-parses; the wiki sits above it, capturing what survived cross-checking and what was walked back.

earth616 — synthesis on top of the extraction pipeline.

Same substrate, same discipline gates, same SPARQL queries — categorically different graph. The rest of the talk shows the direct-ingest mode.

Every LLM session starts from zero.

The problem

Session 1context re-pasted
nothing
carries over
Session 2context re-pasted
nothing
carries over
Session 3context re-pasted


  • ChatGPT and Claude conversations do not remember the last one.
  • Retrieval over your group’s PDFs rediscovers the same chunks on every query.
  • Nothing accumulates between conversations. Nothing compounds between collaborators.

When the student who ran the analysis graduates, so does the knowledge.

Failure paths, preserved not deleted.

Memory that compounds · the file-drawer effect, Rosenthal 1979

Journals publish the win.

The abandoned method, the walked-back claim, the metric bug that inflated an earlier result — all deleted from the record.

Every group has a student who repeated a failed method because no one told them.

The wiki keeps the trajectory.

When an approach turns out wrong, the corrected successor is filed alongside the original with a cross-reference back; the failure stays in place with a Status note pointing forward.

A verification gate ensures new claims land alongside the ones they qualify, not on top of them.

A reader arriving a year later encounters the trajectory, not just the answer.

Case study: the two-author demo branch

Real world example · Web Forager

Chris Frederick pushed a demo branch (commit 81e03e2) for a customer preview and filed the results in a wiki page — Experiment-11-Productionization. The frontmatter carried an extends: edge that would matter later.

---
type: synthesis
up: "[[Experiment-10-Vocabulary-Alignment]]"
related: "[[Experiment-8-Second-Entity]]"
extends: "[[Experiment-7-Question-Deepen]]"
supports: "[[Working-Hypothesis]]"
---

Frontmatter on Experiment-11-Productionization. The extends: edge inherits the analytic frame from Experiment 7.

Reported numbers · Sonnet · 3-entity gold set

Entity Reported
ITAMCO 17 / 17
Southwire 14 / 17
CRC-ND 17 / 17

Ship it?

Chris Sweet opens a separate Claude Code session on main to sanity-check. Two agents, one wiki carrying the state between them.

The reviewer walked the wiki, not the code.

Case study · the finding

Experiment 11
extends:
Experiment 7
extends:
Experiment 8

Same inherited setup across all three.

Following the extends: chain, the reviewer surfaced the mechanism: the pipeline treated the model’s honest “not visible on this page” answer as a successful extraction and stopped the search — exactly the case where deeper searching was warranted.

Reported vs. evidence-based counts

Entity Reported Real
ITAMCO 17 / 17 8 / 17
Southwire 14 / 17 6 / 17
CRC-ND 17 / 17 4 / 17

The bug cascaded across Experiments 7, 8, and 11. The graph, not a debugger, told the reviewer where the fix had to reach.

The LLM refusing to hallucinate looked identical to the LLM answering. The search stopped in exactly the case where deeper searching was warranted.

What made this possible

Case study · what worked

Two authors, two sessions, no coordination.

Both in Claude Code, no handoff meeting — the wiki carried the state.

The typed extends: edge is what made the cascade legible.

Body links would have said “these are related somehow.” The typed edge said “this experiment inherits its analytic frame from that one; if that is wrong, this one probably is too.”

75% coverage recovery after the fix cascaded back through two prior experiments.

Extends chain plus the two criticism edges. Same shape as the demo-branch cascade the reviewer walked.

Full case study: la3d.github.io/WGRA/wiki-collaboration-story/

This does not happen with RAG. Similarity search cannot walk semantic edges backward across experiments.

Team scale · collaborative within a group

The wiki carries the trail across collaborators

Lab notebooks. Slack. Google Docs. Jupyter notebooks. Half-remembered meeting slides. Protocol binders. Thesis drafts.

A postdoc and a grad student on the same project keep notes across all of these in parallel silos. When the PI checks in a month later, most of the trail is scattered or lost.

One attributed wiki instead.

Every log entry carries a by: <human> via <agent> line. Git preserves the amendment history. Concurrent writes: mechanical conflicts resolve automatically; semantic ones route through an LLM-assisted merge.

## [2026-06-30] ingest | Claims-Plus-Walk proposal
- by: Chris Sweet via claude-code
- Created synthesis page [Claims-Plus-Walk]:
  make the atomic claim the retrieval unit for
  MuSiQue; the random walk over a claim graph is
  the mechanism, and entity resolution becomes an
  output of trajectory completion.
- Cross-linked bidirectionally from
  [Bridge-Entity], [Markov-Chain-v5c2],
  [Operations-Basis].
- Testable predictions filed: claim granularity
  gives cleaner trajectory edges than v5c2 (H1);
  claims-plus-walk degrades less on the 281→21k
  jump (H2); ...

Real log entry from the markov_embeddings_and_rag wiki. by: + via: become the audit trail — verifiable in git via git blame on the log file, no separate metadata store, no vendor system.

Team scale · federated across groups

Each group publishes an agent card. Peers discover you by grepping topics.

Comp-chem overlaps with materials science, biophysics, catalysis.

Every group is quietly re-answering questions another campus group already answered last year. When a PI leaves, that group’s institutional memory walks with them.

Each group runs its own wiki, its own permissions. The agent card is how peers find you: grep the topics, discover the capabilities, DM the mailbox.

Three federation modes:

  • ask — clone-and-invoke, synchronous. Shipped in v0.1.0.
  • message — async DMs to a peer’s inbox. Designed, not yet published.
  • post — channel broadcasts to subscribers. Designed, not yet published.
---
type: agent
id: chrissweet/model_fusion
description: Fusion models for pharmaceutical
  discrimination using NIR + PAD imagery.
capabilities:
  - train-cnn-pad-classifier
  - train-pls-r-nir-regressor
  - design-fusion-strategy
topics: [pad-cards, drug-quantitation,
         cnn-classification, nir-spectroscopy]
endpoints:
  inbox: mailbox://chrissweet/model_fusion
---

Agent card from the model_fusion group. The capabilities: and topics: fields are what peers grep against when they need help.

See MVP Eval Run 1 on the team-scale page of the flyer — a five-minute clone-and-invoke federation session across three agents on a real PAD classification task.

Agent capability · built-in workflow

Discipline in the artifact, not in the head of whoever prompts best

Raw group subscription: every student prompts differently. The discipline lives in memory. It walks out with them.

Configured environment: the discipline is in the repo. Every session starts from a known baseline. Any agent that reads .claude/skills/ gets it.

  • Four operations as single commands: wiki-source, wiki-experiment, wiki-lint, plus the verification gate at commit time.
  • SessionStart hook injects the wiki index + last 5 log entries at every session boot. Agent starts oriented, not blank.
  • Discipline gates catch common rationalizations before commit — e.g. “I’ll add the corpus tag later” becomes a warning right when it appears.
---
name: wiki-source
description: Ingest a new source document (paper,
  article, design doc, README, external reference)
  into the wiki. Creates a source-summary page
  and links it into the existing wiki neighborhood.
---

# Procedure (condensed)
1. Read the source; discuss takeaways with user.
2. Create source-summary page. Frontmatter:
   type: source-summary,
   up: closest existing parent page,
   source: URL or filesystem path,
   typed edges (supports:, criticizes:,
   extends:) to related wiki pages.
3. Body: one-sentence opening; sections for
   contribution, methods, intersections, quotes.
4. Update related pages; fix cross-refs in
   both directions on every affected page.
5. Update index; append log entry with
   by: <name> via claude-code.
6. Run Verification Gate at commit time.

The real wiki-source skill file from the template (condensed). Every project instantiated from crcresearch/llm-wiki-memory-template gets the same procedure, model-agnostic.

Agent capability · multi-AI model

Model-agnostic by design

Vendor-locked AI: the memory lives with the vendor. Their subscription changes, your workflow breaks.

Wiki-grounded: the memory is yours in git. Swap the AI.

Different agents read the wiki through their own overlay directory:

  • wiki/agents/claude-code/ — slash commands, hooks
  • wiki/agents/cursor/.mdc rules
  • wiki/agents/opencode/ — coming
  • (Add your own — the pattern is documented)

Same wiki, same schema, same discipline gates. The by: <human> via <agent> line captures which tool did which write.

## [2026-06-14] ingest | Schema extension
- by: Chris Sweet via claude-code

## [2026-06-25] sync | Test harness rename
- by: Chris Sweet via claude-code

## [2026-06-30] ingest | Claims-Plus-Walk
- by: Chris Sweet via claude-code

## [2026-07-04] update | Cursor overlay wired
- by: Chris Sweet via cursor

## [2026-07-05] ingest | New paper section
- by: Priscila Moreira via claude-code

The audit trail records which tool wrote what. When a new model appears, adopting it is a new overlay, not a migration.

One ontology, every wiki: the semantic foundation

Shared substrate · llm-wiki-colab

28 classes, four groups.

  • Core concept · Concept, Entity, Tool
  • Note types · Literature, Method, Theory, Implementation, …
  • Structure pages · Index, MOC, Decision, Finding, …
  • Person / Org

36 semantic properties connect them: extends, supports, criticizes, related, up, authorOf, defines, source, …

The ontology ships at la3d.github.io/llm-wiki-colab. Every wiki instantiated from the template inherits it — no per-project schema work.

SHACL shapes validate structure at commit time. Eleven canned SPARQL queries (hub-notes, ancestors-of, supports-criticizes, …) ship in scripts/kg/sparql/.

Next slide: this ontology applied to a real research wiki — 51 pages, 1,162 triples, live.

The same ontology, a real research wiki

Live instance graph · markov_embeddings_and_rag

markov_embeddings_and_rag

30 nodes, 39 edges. Real research wiki, reified to RDF.

Gold-highlighted edges are the same shape as the web_forager cascade:

  • 4-hop extends chain
  • 2 criticizes edges

Random-Walk-With-Restart — central hub, 48 inbound edges. Surfaced by hub-notes SPARQL, not authored.

No Alt-Tab. Drag nodes to interact.

The pattern across three project types

Same template, three genres of wiki

Web Forager Research execution

Two-author project, typed-edge cascade caught a hedge-counting bug before shipping.

8 of 17 → 17 of 17. 75% coverage recovery.

(the case study you just walked through)

AI-Sci-Disc-Mem External literature survey

115 papers under 20 topic Indexes, 453 typed cross-references.

Chemistry ↔︎ Foundation Models: 18 (the dominant interdisciplinary bond, surfaced by graph query).

The graph shows what actually connects to what.

SavoieAgentKnowledge Group publications catalog

91 papers × 7 research problems. Two dominant problems capture half the work; YARP surfaces as the internal method landmark.

31% of papers still unassigned — the graph knows its own backlog.

Same template, same ontology, same discipline gates. The wiki content is domain-specific.

Getting started: fifteen minutes to a working wiki

Adoption

What ships in v0.1.0

  • crcresearch/llm-wiki-memory-template on GitHub
  • .claude/skills/wiki-source, wiki-experiment, wiki-lint
  • scripts/kg/build-graph.sh, SHACL shapes, 11 SPARQL queries
  • Verification gate + discipline gates preconfigured
  • ask federation mechanism
  • Cursor overlay + minimal-agent mode

Same template as microelectronics-tutor-demo, model_fusion, agent-comms, and the case studies you saw.

Five steps to a working wiki:

  1. Use this template on GitHub, clone locally.
  2. ./scripts/instantiate.sh "My Project"
  3. Ingest your first source — paper, dataset, code repo.
  4. Ask a question. The agent files the answer as a wiki page.
  5. Commit. Every future session starts from that memory.

CRC helps with initial ingest. Bring your first three papers and we’ll walk through the pattern with you.

The wiki is the project. The AI is a tool that reads and writes it.

Learn more · Thanks · Questions

Contact

Chris Sweet · csweet1@nd.edu Center for Research Computing · Notre Dame

Co-authors

Priscila Moreira · Charles Vardeman · James Sweet

Acknowledgements

Chris Frederick (Web Forager collaborator) · CRC leadership · early adopters (markov_embeddings_and_rag, AI-Sci-Disc-Mem, SavoieAgentKnowledge, Web Forager teams)

Questions?