Self-Improving Agents: How Hermes Writes Its Own Skills from Experience

"Self-improving" is one of the most overloaded phrases in AI. In some contexts it means reinforcement learning. In others, fine-tuning. In Hermes, it means something more modest and more auditable: when the agent learns a lesson worth keeping, it writes that lesson as a markdown SKILL.md file and puts it somewhere the next session can read.

The architectural choice is small. The consequences are large. Let's walk through how it works, what changes, and why a markdown file — not a model update — is the right unit of improvement for a working agent.

Key Takeaways

• Hermes self-improvement is procedural-memory growth, not model weight modification. No fine-tuning, no gradient updates.
• When the user corrects the agent, the agent can propose a new SKILL.md capturing the correction, subject to user approval.
• The skill persists in ~/.hermes/skills/ and becomes part of the active skill library on the next session.
• This is additive to FTS5-indexed markdown memory in ~/.hermes/ — memory is facts and context, skills are procedures and patterns.
• Markdown is chosen over opaque formats because skills must be human-readable, reviewable, and diffable.
• The agent decides what to persist, but the user approves. No silent learning.

What "Self-Improving" Means Here

Two separate ideas share this label in the industry, and they are worth disambiguating before going further.

Weight-level self-improvement means the model's parameters change based on experience — typically via RLHF, RLAIF, or continued training. It requires a training pipeline, is expensive, and produces a new model artifact. The model "learns" in the machine-learning sense.

Procedural self-improvement means the model stays fixed, but the context around it accumulates. Instructions, patterns, and counter-examples pile up in a place the model will read next time. The model "learns" only in the colloquial sense — really, its situation learns.

Hermes does the second thing. The Claude model powering the agent is the same model every session. What changes is the library of skills and the memory files the agent gets loaded alongside the user's prompt.

The Core Flow

Here is a concrete sequence. The user is working on a Python project and asks Hermes to write tests.

1. Turn 1: Agent writes tests, commits them, moves on.
2. Turn 2: User says, "Wait, you wrote the implementation first. I want tests written before implementation, failing, then I want to see you make them pass."
3. Agent response: The agent notices this is not a one-off preference — it is a general working pattern the user wants applied to future work. It proposes: "Should I capture this as a skill named test-driven-development so I follow this pattern automatically?"
4. User approves. Agent writes the SKILL.md.
5. Turn 3, next session: User asks for a new feature. The skill selector matches on the task description. The skill activates. The agent writes the failing test first, unprompted.

The agent did not learn in the neural sense. It wrote itself a note, and the next instance of itself read the note.

What the Written Skill Looks Like

The verbatim frontmatter of the Hermes test-driven-development skill — which originated through exactly this flow when the Hermes team adapted it from the obra/superpowers library:

---
name: test-driven-development
description: Use when implementing any feature or bugfix, before writing implementation code. Enforces RED-GREEN-REFACTOR cycle with test-first approach.
version: 1.1.0
author: Hermes Agent (adapted from obra/superpowers)
license: MIT
metadata:
  hermes:
    tags: [testing, tdd, development, quality, red-green-refactor]
    related_skills: [systematic-debugging, writing-plans, subagent-driven-development]
---

Three things worth noticing. First, the description is activation guidance for the skill selector — the more precisely worded, the more reliably the skill fires when relevant. Second, related_skills links this skill into a network; when TDD activates, the agent knows there are other skills nearby if debugging is needed. Third, the version is semver, so the skill can evolve across sessions without destabilizing earlier agreed behavior.

Why Markdown

The choice of format matters more than it looks. Three reasons markdown beats opaque alternatives:

• Inspectable. You can read what your agent believes. Compare to an agent whose "learning" is buried in weights you cannot introspect.
• Auditable. Skills live in git. A diff shows exactly what changed and when.
• Editable. If the agent wrote a skill that is subtly wrong, you open the file and fix it. No retraining, no data pipeline.

The trade-off is obvious: markdown skills do not generalize the way weight updates do. If the agent learns "prefer subprocess.run with an argv list over shell-string APIs," a markdown skill captures that as a written instruction, not as a learned bias that automatically transfers to similar-but-different situations. That is a real limitation, and it is why skills are procedural — they work for patterns the user can articulate, not for statistical regularities that cannot be named.

Skills vs Memory

Hermes has two parallel persistence surfaces. They are often conflated.

• Memory: FTS5-indexed markdown files in ~/.hermes/. The agent writes entries about facts, context, and session summaries. Retrieval is full-text search.
• Skills: Named SKILL.md files under ~/.hermes/skills/. Activation is via the skill selector matching on the description field.

Memory answers: "What do I know about this user, this project, this conversation so far?"

Skills answer: "When I encounter task X, how do I do it?"

Both are written in markdown. Both the agent can author. The distinction is that memory is contextual (facts about the world) and skills are procedural (how to do a class of things). See Hermes Memory Deep Dive for the memory side.

The User-Approval Gate

A key detail: Hermes does not silently persist. When the agent decides a pattern is skill-worthy, it proposes. The user approves, edits, or rejects. This matters for three reasons.

1. Not every correction is a general rule. Sometimes the user wants a one-off, not a pattern.
2. The agent's phrasing may be wrong. Reading the proposed SKILL.md lets the user fix the description before it ships.
3. Trust. An agent that quietly modifies its own behavior is an agent you cannot reason about. The approval step keeps the loop legible.

Where This Leads

The practical consequence is that a Hermes instance used regularly by one person starts to feel like an apprentice. After a few weeks, its bundled skills are augmented by a personalized library — the user's preferred commit style, deployment checklist, PR review rubric, triage flow. None of that required training. All of it is inspectable.

For the adjacent architectural idea — the agent delegating work to subagents — see Spawning Claude Code as a Hermes Subagent.

Sources

• Hermes Agent repository — https://github.com/NousResearch/hermes-agent
• Hermes documentation — https://hermes-agent.nousresearch.com/docs/
• obra/superpowers reference implementation — https://github.com/obra/superpowers
• Anthropic Claude documentation — https://docs.anthropic.com/claude
• Series: Hermes Memory Deep Dive: FTS5 Markdown Recall
• Series: The agentskills.io Standard
• Series: Spawning Claude Code as a Hermes Subagent