Cosma: Core Agent Functionality for LLM-Powered Agents

Core Agent functionality for building LLM-powered agents with answer.ai cosette and OpenAI API. Cosma is a library for building LLM-powered agents using cosette. It provides: - A simple Agent class for creating tool-using LLM agents - Production-ready logging and metrics collection - Support for Anthropic’s recommended workflow patterns

Author

Charles F. Vardeman II

What is Cosma?

Cosma is a library for building LLM-powered agents using Answer.AI’s cosette library for interacting with the OpenAI API. Answer.AI also built cosette’s sister claudette for Anthropic APIs. It extends the cosette core Chat and Toolloop to support building agents using the Anthropic Approach to “Building Effective Agents”. The markdown for the guide is in the repositories ctxs folder to be provided as a context for a LLM to help build agents. This library was built using dialog engineering and the SolveIt! dialogs are in the dialogs folder as ipynb files. The nbsanity service is useful for rendering the notebooks in a more readable format.

It provides:

  1. A simple Agent class for creating tool-using LLM agents
  2. Production-ready logging and metrics collection
  3. Support for Anthropic’s recommended workflow patterns
Note

Working with dialog engineering in VSCode is more challenging than in the SolveIt! notebooks. This package will eventually provide “Custom instructions for GitHub Copilot in VS Code” to assist in building agents using a nbdev approach and Github Copilot Chat approach.

Quick Start

from cosma.core import Agent
from cosma.logging import AgentLogger
import math

# Define a tool
def solve_math(
    expression: str  # Mathematical expression to evaluate
) -> float:         # Numerical result
    """Evaluates mathematical expressions using Python's math module."""
    return eval(expression, {"__builtins__": {}}, 
               {"sqrt": math.sqrt, "pi": math.pi})

# Create an agent
agent = Agent(
    role="math tutor",
    model="gpt-4o",
    tools=[solve_math]
)

# Use the agent
response = agent.run_with_tools("What is sqrt(16) + 7?")

Core Features

  1. Simple Agent Creation
    • Define roles and tools
    • Automatic conversation management
    • Token usage tracking
  2. Production Logging
    • Structured JSON logging
    • Metrics collection
    • Console and file outputs
  3. Tool Integration
    • Type-hinted tool definitions
    • Automatic tool documentation
    • Safety constraints

Motivation

Cosma emerged from the need to build production-ready LLM agents that combine: - Fast.ai’s exploratory programming approach - Anthropic’s agent development best practices - Production-ready logging and deployment capabilities

Exploratory Development in Solve It

Solve It provides an ideal environment for agent development through: 1. Interactive REPL-driven development 2. Real-time testing of agent-tool interactions 3. Immediate feedback on prompt engineering 4. Literate programming with markdown documentation

This approach allows rapid iteration on: - Tool design and documentation - System prompts and role definitions - Conversation flow patterns - Error handling and edge cases

Anthropic’s Agent Patterns

Cosma implements key insights from Anthropic’s “Building Effective Agents” guide:

  1. Workflows vs Agents
    • Workflows: Predefined orchestration of LLMs and tools
    • Agents: Systems where LLMs dynamically direct their process
    • Cosma supports both through run() and run_with_tools()
  2. Common Patterns
    • Prompt Chaining: Breaking tasks into sequential steps
    # Example: Two-step math solution
outline = agent.run("Outline steps to solve: 3x + 7 = 22")
solution = agent.run(f"Now solve step by step:\n{outline}")
    • Routing: Classifying and directing inputs
    # Example: Route to specialized agents
agents = {
    'math': math_agent,
    'writing': writing_agent
}
task = router_agent.run(prompt)
response = agents[task].run(prompt)
    • Parallelization: Running subtasks simultaneously
    # Example: Multiple perspectives
responses = [
    agent.run(prompt) for _ in range(3)
]
synthesis = agent.run(f"Synthesize these views:\n{responses}")
    • Evaluator-Optimizer: Iterative refinement loops
    # Example: Iterative improvement
draft = agent.run(prompt)
feedback = evaluator.run(draft)
final = agent.run(f"Improve based on:\n{feedback}")
  3. Best Practices
    • Start simple, add complexity only when needed
    • Use clear, well-documented tools
    • Maintain traceability through logging
    • Set appropriate safety limits

Production Considerations

Cosma addresses key production needs:

  1. Structured Logging

    logger = AgentLogger("math_agent", log_dir="/var/log/agents")
logger.log_event("tool_called", 
    tool="solve_math",
    input="sqrt(16)",
    result=4.0
)

  2. Metrics Collection

    • Token usage tracking
    • Response times
    • Tool usage patterns
    • Success/failure rates

  3. Safety Features

    • Maximum steps limits
    • Tool sandboxing
    • Input validation
    • Error handling

  4. Deployment Ready

    • Container-friendly logging
    • Resource monitoring
    • State management
    • Error recovery

Why Cosma?

Traditional agent frameworks often: - Hide implementation details - Force specific architectural patterns - Make debugging difficult - Limit customization

Cosma instead provides: 1. Transparent, composable building blocks 2. Full control over agent behavior 3. Clear logging and monitoring 4. Production-ready features

This approach enables: - Rapid prototyping in notebooks - Easy transition to production - Clear debugging and testing - Flexible architecture choices

Developer Guide

If you are new to using nbdev here are some useful pointers to get you started.

Install cosma in Development mode

# make sure {{lib_path}} package is installed in development mode
$ pip install -e .

# make changes under nbs/ directory
# ...

# compile to have changes apply to {{lib_path}}
$ nbdev_prepare

UV instructions

# Install uv
curl -LsSf https://astral.sh/uv/install.sh | sh

# Create and activate a virtual environment
uv venv
source .venv/bin/activate

# Install the project in editable mode
uv pip install -e ".[dev]"

# Install additional dependencies
uv pip install nbdev jupyter

Installation

Install latest from the GitHub repository:

$ pip install git+https://github.com/la3d/cosma.git

Documentation

Documentation can be found hosted on this GitHub repository’s pages. Additionally you can find package manager specific guidelines on and pypi respectively.