hermes-agent/memory-bank/activeContext.md

Active Context

Current Focus

Consolidating the two Atropos environment systems and fixing tool calling to use proper OpenAI-spec approach instead of ICL.

PR Feedback from Lead Dev (Feb 10, 2026)

The PR was rejected because our approach has three fundamental issues:

Issue 1: ManagedServer doesn't pass tools={} to apply_chat_template()

• When using Phase 2 (VLLM/SGLang for RL training), ManagedServer needs to pass tools to tokenizer.apply_chat_template(tools=...)
• This makes the system prompt include tool definitions the way models were trained to expect
• Fix: Atropos PR #366 adds tool_call_parser support to ManagedServer (branch: tool_call_support)

Issue 2: ICL prompt vs proper tool calling

• Our code embeds tools as XML in the system prompt (<tools>...</tools>)
• Proper approach: pass tools= parameter in chat_completion() calls and let the tokenizer's chat template handle formatting
• All Hermes datasets train on the proper format, not ICL

Issue 3: Only Hermes <tool_call> parser, no multi-model support

• Our code only handles Hermes-style <tool_call> XML parsing
• Proper approach: parser registry supporting 11+ model families (hermes, qwen, deepseek, llama, mistral, etc.)

Architecture: What Exists Now (Two Parallel Systems)

environments/ (Teknium's proper approach) ✅ CORRECT

environments/
├── agent_loop.py              ← Uses tools= in chat_completion() (OpenAI spec)
├── hermes_base_env.py         ← Phase 1 (OpenAI) + Phase 2 (ManagedServer + parser)
├── tool_context.py            ← ToolContext for reward functions
├── tool_call_parsers/         ← 11 model parsers (hermes, qwen, deepseek, llama, etc.)
│   ├── __init__.py            ← Registry with get_parser(), register_parser()
│   ├── hermes_parser.py
│   ├── qwen_parser.py
│   ├── deepseek_v3_parser.py
│   ├── llama_parser.py
│   ├── mistral_parser.py
│   └── ... (11 total)
├── terminal_test_env.py       ← Working example: file creation tasks
├── hermes_swe_env.py          ← SWE environment
└── patches.py                 ← Async-safe monkey patches

How it works correctly:

1. HermesAgentLoop.run() passes tools=self.tool_schemas to chat_completion()
2. ManagedServer passes tools to tokenizer.apply_chat_template(tools=...)
3. Parser registry reconstructs tool_calls from raw model output
4. Tool execution uses hermes-agent's handle_function_call() from model_tools.py

atropos/ (Our sandbox-optimized code) - PARTIALLY REDUNDANT

atropos/
├── agent/atropos_agent.py     ← ICL-based agent (REDUNDANT with agent_loop.py)
├── envs/agent_env.py          ← Environment with sandbox backends (PARTIALLY REDUNDANT)
├── envs/swe_smith_oracle_env.py ← SWE env using sandbox (KEEP - port to new base)
├── backends/                  ← Sandbox backends (KEEP - valuable infrastructure)
│   ├── modal_backend.py       ← Modal sandbox pool
│   ├── nomad_backend.py       ← Nomad/Docker/Singularity
│   └── base.py                ← ToolBackend protocol
├── slots/                     ← Slot multiplexing (KEEP)
├── nomad/                     ← Nomad client (KEEP)
├── tools/                     ← Sandbox tool registry (PARTIALLY REDUNDANT)
└── sandbox_server.py          ← HTTP server in containers (KEEP)

Plan: Consolidate into environments/

What to KEEP from atropos/:

• backends/ - Modal, Nomad, Singularity backends (valuable infrastructure for scale)
• slots/ - Slot multiplexing
• nomad/ - Nomad client
• sandbox_server.py - Container HTTP server
• Dockerfile - Sandbox container image

What to REMOVE/REPLACE:

• atropos/agent/atropos_agent.py → replaced by environments/agent_loop.py
• atropos/envs/agent_env.py → functionality merged into environments/hermes_base_env.py
• atropos/tools/ → replaced by model_tools.py + tools/ (hermes-agent's standard tools)

What to CREATE:

• environments/gsm8k_agent_env.py → GSM8k with tool calling, subclasses HermesAgentBaseEnv
• Update environments/hermes_base_env.py to optionally use sandbox backends (Nomad/Modal) for terminal isolation when needed for scale

Steps:

1. Install atropos tool_call_support branch (PR #366)
2. Create environments/gsm8k_agent_env.py using HermesAgentBaseEnv
3. Port swe_smith_oracle_env.py to use HermesAgentBaseEnv
4. Make sandbox backends accessible from HermesAgentBaseEnv (terminal_backend config)
5. Remove redundant atropos/agent/ and atropos/envs/agent_env.py
6. Clean up atropos/tools/ (keep only sandbox-specific tools)
7. Update tinker-atropos gsm8k env to use proper base class
8. Test everything end-to-end

Previous Completed Work

• Modal backend integration (Feb 8) - KEEP backends, update integration point
• Main branch merge (Feb 9) - completed
• Singularity/Apptainer (Feb 6) - KEEP
• Memory Bank initialized (Feb 5)