AI4C2 · research foundations

Beyond the Model

Harnesses and Architectures for Language Agents

Charles F. Vardeman II
Center for Research Computing, University of Notre Dame
August 18, 2026
Language models

An LLM predicts a continuation

A large language model maps the tokens already in its context to probabilities for what token should come next.

Context

Instructions, examples, retrieved information, and the conversation so far.

Model

A trained neural network transforms that context into a distribution over tokens.

Continuation

Sampling one token at a time produces text—and can request structured tool calls.

Further viewing

Build the full mental model—from tokens to ChatGPT

Video thumbnail showing Andrej Karpathy beside a visualization of the computations inside ChatGPT
Andrej Karpathy · recommended introduction

Deep Dive into LLMs like ChatGPT

  • How text becomes tokens and next-token predictions
  • How pretraining and post-training shape behavior
  • Why tools and agent loops extend the base model

Watch: “Deep Dive into LLMs like ChatGPT” · long-form visual introduction.

Tool execution

The model requests an action; the harness executes it

A language model emits a structured tool call; the harness validates and invokes an ordinary software function, then returns the result as context

The LLM proposes the call. The harness mediates every crossing of the language–software boundary.

From generation to action

An agent puts tools in a loop

“An LLM agent runs tools in a loop to achieve a goal.”

1 · Goal

A bounded objective and a condition for stopping.

2 · Tools

Actions that retrieve information or change an external environment.

3 · Feedback

Tool results return as observations that shape the next model call.

Definition: Simon Willison, “I think ‘agent’ may finally have a widely enough agreed upon definition.”

Training dynamics

Memorization and generalization can arrive at different times

Animated grokking example: training accuracy rises quickly while test accuracy remains near chance, then test accuracy rises sharply after prolonged training

Animation: Pearce et al., “Do Machine Learning Models Memorize or Generalize?” · phenomenon: Power et al., “Grokking.”

Inference-time adaptation

Examples in the prompt teach a temporary task

Task · sentiment classification

Decide whether a review expresses a positive or negative opinion.

PROMPT

Classify each review as POSITIVE or NEGATIVE.

I loved this movie.POSITIVE
This was terrible.NEGATIVE
Surprisingly enjoyable.?
→
MODEL COMPLETION POSITIVE

No retraining: the examples shape behavior only while they remain in the prompt.

In-context learning: Brown et al., “Language Models are Few-Shot Learners” · Garg et al., “What Can Transformers Learn In-Context?”

Inside the transformer

An induction head finds a pattern and continues it

An attention head is a small part of a transformer that decides which earlier words are useful for predicting the next one.

EARLIER IN THE PROMPT The code word is BLUEBIRD.
look back at what followed the same words
LATER IN THE PROMPT The code word is → BLUEBIRD
1Find a familiar sequence
2See what followed it earlier
3Use that continuation again

Mechanistic evidence: Olsson et al., “In-context Learning and Induction Heads.”

Continue learning

Three visual explanations worth watching

Welch Labs video thumbnail showing neural-network layers and a next-token prediction 1 · How training works

The F=ma of Artificial Intelligence

Backpropagation and how training changes a model’s weights.

 Welch Labs video thumbnail titled The Geometry of Depth 2 · Why depth helps

Why Deep Learning Works Unreasonably Well

A visual account of deep networks and generalization.

 Welch Labs video thumbnail showing geometric patterns around the word Grokking 3 · Looking inside

The Most Complex Model We Actually Understand

Grokking and a rare mechanistic explanation of a learned algorithm.

All three videos: Welch Labs · companion grokking animation

Statistical-mechanics aside

Grokking may resemble slow glass relaxation—not a first-order transition

Treat the neural network as a physical system: its parameters are the degrees of freedom, and training loss plays the role of energy.

Glass physicsatoms rearrange
LIQUIDmobile configurations
rapid quench→
NONEQUILIBRIUM GLASSmotion becomes sluggish
slow relaxation→
STABLE CONFIGURATIONmore equilibrated
Neural networkparameters change
EARLY TRAININGmany possible solutions
fast optimization→
MEMORIZATIONlow loss; poor test accuracy
continued training→
GENERALIZATIONhigh test accuracy
No entropy barrier observed The sampled landscape is continuous between memorizing and generalizing states—evidence against a first-order transition in these experiments.

Zhang et al., “Is Grokking a Computational Glass Relaxation?” NeurIPS 2025 · scope: one-layer transformers on modular-arithmetic tasks.

From model to system

Capability is moving outward

Three layered stages of language-agent capability: model weights, assembled context, and harness infrastructure

Adapted from Zhou et al., “Externalization in LLM Agents,” Fig. 2 and §2.

Externalization

Reliable agency relocates recurring burdens

Memory

Recall → recognition

Persist state outside the model and retrieve what the present decision needs.

Skills

Generation → composition

Package procedures so workflows need not be improvised from scratch.

Protocols

Ad hoc → structured

Turn ambiguous interaction into machine-readable, governable exchange.

Organizing principle: Zhou et al., “Externalization in LLM Agents,” §1.

Externalized state

Memory carries continuity across time

Working context

Plans, open files, hypotheses, and checkpoints for the active task.

Episodic experience

Prior trajectories, decisions, failures, outcomes, and reflections.

Semantic knowledge

Stable facts, project conventions, abstractions, and domain guidance.

Personalized memory

User- or environment-specific preferences with distinct retention rules.

Sources: Zhou et al., §3 and Fig. 4 · Anthropic, “Effective context engineering for AI agents.”

Externalized expertise

Skills make procedures reusable and inspectable

Operational procedure

Steps, dependencies, recovery paths, and stopping conditions.

Decision heuristics

Rules for choosing tools, strategies, or branches under recurring conditions.

Normative constraints

Required checks, prohibited actions, quality standards, and escalation rules.

authoreddistilleddiscoveredcomposed→ registry → progressive disclosure → execution

Source: Zhou et al., §4 and Fig. 5.

Externalized interaction

Protocols govern how capabilities cross boundaries

Invocation grammar

Arguments, types, ordering, result shape, and validation.

Lifecycle semantics

Allowed transitions, turn ownership, completion, and failure.

Permission and trust

Who may act, what data may move, and which evidence is required.

Discovery metadata

Registries and capability descriptions make interfaces findable.

Tools expose operations · skills encode how to use them · protocols govern interaction. Zhou et al., §5 and Fig. 6.

Harness engineering

The harness is a designed cognitive environment

A foundation model surrounded by memory, skills, protocols, permission, control, and observability within a harness

Adapted from Zhou et al., Fig. 7 and §6.

Harness design

Six dimensions define the operating envelope

Loop and control

Steps, branches, recursion, termination, and costs.

Sandboxing

Filesystem, network, state, and execution isolation.

Human oversight

Approval gates, review points, and escalation triggers.

Observability

Traces, metrics, causal links, errors, and outcomes.

Policy encoding

Versioned permissions across user, project, and organization scopes.

Context budgets

Retrieval, loading, compaction, eviction, and allocation.

Analytical framework from Zhou et al., §6.2.

At the model boundary

The harness structures model input—and output

INPUT

Memory

Selected historical and situational context.

contextual input

Skills

Procedures, examples, heuristics, and constraints.

instructional input
LLM

Protocols

Typed calls constrain the generative action space.

action schema
OUTPUT

Separation makes retrieval, procedure, and interface failures independently debuggable. Zhou et al., §7.2.

Architectural bridge

CoALA organized cognition; externalization locates its machinery

CoALA · 2023/2024

Functional vocabulary

Memory modules, internal and external actions, and a decision cycle explain what a language agent does.

Externalization · 2026

Systems partition

Memory, skills, and protocols explain where state, expertise, and interaction structure live—and how the harness governs them.

Sources: Sumers et al., “Cognitive Architectures for Language Agents” · Zhou et al., “Externalization in LLM Agents.”

Multi-step reinforcement learning

Agentic RL trains over trajectories—not single responses

Agentic reinforcement learning loop in which a policy model produces policy steps, an orchestration harness dispatches tool calls into a sandbox environment, observations return as environment steps, and the completed trajectory is graded for reward

Adapted from Microsoft AI Team, MAI-Thinking-1, Fig. 18 and §3.3, p. 40.

Harness reinforcement learning

RL can operate through—or on—the harness

RL through the harness

The runtime owns the rollout while training changes the model policy that emits reasoning, tool calls, and final answers.

RL of the harness

A controller learns structural choices—retrieve, branch, check, retry, compact, or stop—while the executor may remain frozen.

Sources: Agent Lightning v1.0 · Learning to Control LLM Agent Harnesses with Offline RL

Reasoning vocabulary

Two ways to move from evidence to a conclusion

Induction · examples → likely rule
Copper expands when heated.Aluminum expands when heated.Steel expands when heated.
↓
Metals probably expand when heated.
Deduction · rule + fact → necessary conclusion
Rule: All metals expand when heated.Fact: Copper is a metal.
↓
Copper will expand when heated.

Useful division of labor: language models recognize patterns and propose possibilities; symbolic reasoners apply explicit rules and return checkable consequences.

Recursive Language Models

A long prompt is not organized evidence

Imagine answering one question across a 500-page report archive. Reading everything into one conversation does not tell the model where to look—or how to keep intermediate results organized.

Store it as data

Keep the full archive outside the model’s immediate conversation.

Search before reading

Inspect and select the portions relevant to the present question.

Ask smaller questions

Solve bounded subproblems and combine their results in code.

Source: Zhang, Kraska, and Khattab, “Recursive Language Models,” v3.

Recursive Language Models

An RLM turns the prompt into a working environment

Root model = coordinator · REPL workspace = external notebook with variables and code

Context is offloaded into symbolic variables and programmatic sub-agent calls keep task-specific intermediate results outside the root model context

Adapted from Zhang and Khattab, “Language model harnesses are compositional generalizers,” Fig. 5.

Inductive bias

Structure makes unfamiliar problems feel familiar

Inductive reasoning infers a pattern from examples. An inductive bias is built-in structure that makes some solutions easier to find.

A complex out-of-distribution task is transformed by a structured harness into several small locally in-distribution observations for language-model calls

Concept and early evidence: Zhang and Khattab, “Language model harnesses are compositional generalizers,” Figs. 1–5.

Symbolic reasoning as a tool

An RLM can ask a reasoner to derive what follows

FACT :Socrates a :Man.
+
RULE Man(x) → Mortal(x)
→
EYELENG TOOL apply the rule forward or backward reasoning
→
DERIVED + PROOF :Socrates a :Mortal.

The guarantee is conditional: the derivation is valid only if the translated facts and rules are correct.

Example and capabilities: Eyeleng—hybrid forward materialization, backward proving, validation, and proof explanations.

Neurosymbolic RLM

A neurosymbolic agent separates responsibilities

Neural modelInterpret language · recognize patterns · propose facts and actions
↓
RLM harnessManage context · choose tools · recurse · preserve provenance
↓
Symbolic reasonerApply rules · validate constraints · return conclusions and proofs

Design synthesis; tools and examples: Eyeleng · Coyle, “Why Agentic Systems Need Ontologies” · Albinhassan et al.

Further viewing

Why agentic systems need ontologies

Video thumbnail featuring Frank Coyle with the words Ontologies Keep Agents Honest
Frank Coyle · UC Berkeley · AI Engineer

Probabilistic reasoning inside.
Logical guardrails outside.

  • 4:23 · neurosymbolic AI
  • 9:19 · RDFS and OWL inference
  • 14:23 · validation inside an agent loop
  • 17:43 · type safety vs. domain correctness

Watch: “Why Agentic Systems Need Ontologies” · 21 minutes.

Case study · PRIME Agent

PRIME Agent makes an RLM persistent

The model can reorganize information, run code, delegate work, and retain useful lessons—without changing its weights during the task.

MODEL INVOCATION
L0Model weightslearned before the task
L1Active contextwhat the model sees now
explicitly managed state begins here→
PRIME AGENT HARNESS
L2Persistent REPL + recursive agentscompute, tools, and parallel subproblems
L3History + memories + skillsretained, versioned, and reusable
REPORTED RESULT30% → 95.5%ARC-AGI-3 RHAE Best@1
LONG HORIZON85.5 hoursone autonomous nanoGPT run
DESIGN WARNINGPersistence remembers shortcuts, tooleast privilege, independent validation, and rollback still matter

Karten et al., “Prime Agent: A Self-Improving RLM Harness” (preprint, Aug. 2026) · open-source implementation.

AI4C2 starting experiment

Compare externalization strategies—not only models

System What is externalized Research question
Base LLM Nothing beyond one assembled prompt Where do quality and traceability degrade?
Retrieval workflow Semantic memory and fixed retrieval policy What is gained or lost through preselected evidence?
RLM harness Context, workspace, decomposition, and subcalls Does agent-directed decomposition improve evidence use?
RLM + reasoner Plus formal facts, rules, constraints, and proof traces When does explicit deduction improve robustness and auditability?

Public or synthetic C2-style artifacts · measures: quality, provenance, context cost, latency, failures, and recovery.

Research question

What contract should connect an LLM agent to a symbolic reasoner?

How should an agent formalize evidence, invoke deduction, and consume proof-bearing results without hiding errors at the neural–symbolic boundary?

1 · FORMALIZE Language → formal claims Typed facts · rules · query · source provenance
→
2 · REASON Derive what follows Conclusion · proof · contradiction · invalid · unknown
→
3 · ACT Control downstream use Validate provenance and policy before planning or tool use

Test against an RLM-only baseline: Does the interface improve correctness, contradiction detection, and auditability at acceptable cost and latency?

Research target: the schema, provenance, failure semantics, and control policy at the neural–symbolic boundary.

AI4C2 · Agentic AI for C2 Platforms