mirrors/hermes-agent
1
0
Fork 0
mirror of https://github.com/NousResearch/hermes-agent.git synced 2026-06-12 08:51:53 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 14

fix(agent): demote non-coding skill categories to names-only — never hide skills (#44342)

Browse source 
Real-world failure with the original index pruning: under the default auto
posture, an agent-created ops skill in a demoted category vanished from the
prompt's skill index mid-project, and the agent silently fell back to a
stale sibling skill instead. The "discovery-only" premise didn't hold —
models do not reach for skills_list to rediscover what the index stops
showing them, and agent-created skills are the model's accumulated project
memory (runbooks, pitfalls, operating rules).

Gating pruning behind the opt-in focus mode was the wrong fix too: users
opening a worktree don't know the config exists, so the index-noise win
would effectively never ship.

Instead, the coding posture now DEMOTES non-coding categories rather than
hiding them: each demoted category renders as a single names-only line
("gaming [names only]: allthemons10-ops, mc-backup") with a footer note
explaining the omitted descriptions. Every skill name stays in the prompt,
so memory-anchored recall ("load <name>") keeps working in every mode,
while the description noise is still cut. Applies in auto/on/focus alike;
the general posture demotes nothing. Deny-list semantics unchanged —
unknown/custom categories and coding-adjacent ones keep full entries.

API renamed to match the honest semantics: hidden_skill_categories →
compact_skill_categories, build_skills_system_prompt(hidden_categories=) →
compact_categories=.
This commit is contained in:
brooklyn! 2026-06-11 10:25:42 -05:00 • committed by GitHub
parent 9c051f57c3
commit ee1a744ace
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
5 changed files with 122 additions and 84 deletions
Download patch file Download diff file Expand all files Collapse all files

61
agent/coding_context.py
View file

@ -38,9 +38,10 @@ session (deferred), the same contract as ``/skills install`` vs ``--now``.
Activation (config ``agent.coding_context``):
* ``auto`` (default) — posture (brief + snapshot) on an interactive coding
surface sitting in a code workspace (git repo or recognised project root).
Prompt-only; toolsets untouched.
* ``auto`` (default) — posture (brief + snapshot + names-only demotion of
non-coding skill categories) on an interactive coding surface sitting in
a code workspace (git repo or recognised project root). Prompt-only;
toolsets untouched, no skill is ever hidden.
* ``focus`` — like ``auto``, but additionally collapses the toolset to the
``coding`` set + enabled MCP servers. Explicit opt-in for a lean schema.
* ``on`` — force the posture anywhere (incl. non-workspaces). Prompt-only.
@ -212,11 +213,13 @@ class ContextProfile:
``model_hint`` — routing preference key for smart model routing
(extension seam; not yet consumed by the router).
``memory_policy``— memory namespace/weighting hint (extension seam).
``hidden_skill_categories`` — skill categories pruned from the system-prompt
skill index while this posture is active. Discovery-only:
nothing is disabled — ``skills_list`` still returns the
full catalog and ``skill_view`` loads anything. Deny-list
semantics so unknown/custom categories stay visible.
``compact_skill_categories`` — skill categories DEMOTED to names-only in
the system-prompt skill index while this posture is
active. Never hidden: every skill name stays visible
(so memory-anchored recall keeps working) — only the
descriptions are dropped to cut index noise. Deny-list
semantics so unknown/custom categories keep full
entries.
"""
name: str
@ -224,14 +227,14 @@ class ContextProfile:
guidance: str = ""
model_hint: Optional[str] = None
memory_policy: str = "default"
hidden_skill_categories: tuple[str, ...] = ()
compact_skill_categories: tuple[str, ...] = ()
# Skill categories that are clearly not part of a coding workflow. Hidden from
# the prompt's skill index in the coding posture (deny-list — anything not
# listed here, incl. custom user categories, stays visible). Coding-adjacent
# categories (devops, github, mcp, data-science, diagramming, research,
# security, …) are intentionally absent.
# Skill categories that are clearly not part of a coding workflow. Demoted to
# names-only in the prompt's skill index while the coding posture is active
# (deny-list — anything not listed here, incl. custom user categories, keeps
# full entries). Coding-adjacent categories (devops, github, mcp,
# data-science, diagramming, research, security, …) are intentionally absent.
_NON_CODING_SKILL_CATEGORIES = (
"apple", "communication", "cooking", "creative", "email", "finance",
"gaming", "gifs", "health", "media", "music", "note-taking",
@ -247,7 +250,7 @@ CODING_PROFILE = ContextProfile(
guidance=CODING_AGENT_GUIDANCE,
model_hint="coding",
memory_policy="project",
hidden_skill_categories=_NON_CODING_SKILL_CATEGORIES,
compact_skill_categories=_NON_CODING_SKILL_CATEGORIES,
)
_PROFILES: dict[str, ContextProfile] = {
@ -432,9 +435,20 @@ class RuntimeMode:
blocks.append(workspace)
return blocks
def hidden_skill_categories(self) -> frozenset[str]:
"""Skill categories to prune from the prompt's skill index (may be empty)."""
return frozenset(self.profile.hidden_skill_categories)
def compact_skill_categories(self) -> frozenset[str]:
"""Skill categories to demote to names-only in the prompt's skill index.
Demoted — never hidden. An earlier revision fully pruned these
categories from the index, which caused silent capability loss in a
real workflow: agent-created skills are the model's accumulated
project memory (server-ops runbooks, learned pitfalls, …), and models
do not reliably reach for ``skills_list`` to rediscover what the
index stopped showing them. Names-only keeps every skill loadable on
recall while still cutting the description noise from the index.
"""
if not self.is_coding:
return frozenset()
return frozenset(self.profile.compact_skill_categories)
def resolve_runtime_mode(
@ -512,20 +526,21 @@ def coding_system_blocks(
).system_blocks()
def coding_hidden_skill_categories(
def coding_compact_skill_categories(
*,
platform: Optional[str] = None,
cwd: Optional[str | Path] = None,
config: Optional[dict[str, Any]] = None,
) -> frozenset[str]:
"""Skill categories the active posture prunes from the prompt's skill index.
"""Skill categories the active posture demotes to names-only in the index.
Empty outside the coding posture. Discovery-only: hidden skills remain
loadable via ``skills_list`` / ``skill_view``.
Empty outside the coding posture. Demoted — never hidden: every skill
name stays in the index and remains loadable via ``skill_view`` /
``skills_list``; only descriptions are dropped.
"""
return resolve_runtime_mode(
platform=platform, cwd=cwd, config=config
).hidden_skill_categories()
).compact_skill_categories()
def _enabled_mcp_servers(config: Optional[dict[str, Any]]) -> list[str]:

60
agent/prompt_builder.py
View file

@ -1101,7 +1101,7 @@ def _skill_should_show(
def build_skills_system_prompt(
available_tools: "set[str] | None" = None,
available_toolsets: "set[str] | None" = None,
hidden_categories: "frozenset[str] | None" = None,
compact_categories: "frozenset[str] | None" = None,
) -> str:
"""Build a compact skill index for the system prompt.
@ -1117,11 +1117,11 @@ def build_skills_system_prompt(
are read-only — they appear in the index but new skills are always created
in the local dir. Local skills take precedence when names collide.
``hidden_categories`` (e.g. from the coding posture — see
agent/coding_context.py) prunes whole categories from the rendered index.
Discovery-only: the snapshot stores everything, ``skills_list`` /
``skill_view`` still reach every skill, and a footer note tells the model
the full catalog exists.
``compact_categories`` (e.g. from the coding posture — see
agent/coding_context.py) demotes whole categories to a names-only line in
the rendered index. Nothing is ever hidden: every skill name stays
visible and loadable via ``skill_view`` / ``skills_list``; only the
descriptions are dropped, and a footer note explains the demotion.
"""
skills_dir = get_skills_dir()
external_dirs = get_all_skills_dirs()[1:] # skip local (index 0)
@ -1146,7 +1146,7 @@ def build_skills_system_prompt(
tuple(sorted(str(ts) for ts in (available_toolsets or set()))),
_platform_hint,
tuple(sorted(disabled)),
tuple(sorted(hidden_categories or ())),
tuple(sorted(compact_categories or ())),
)
with _SKILLS_PROMPT_CACHE_LOCK:
cached = _SKILLS_PROMPT_CACHE.get(cache_key)
@ -1280,38 +1280,44 @@ def build_skills_system_prompt(
except Exception as e:
logger.debug("Could not read external skill description %s: %s", desc_file, e)
# Posture-driven category pruning (e.g. non-coding skills while pairing on
# code). Match on the top-level category segment so nested categories
# ("social-media/twitter") are pruned with their parent.
# Posture-driven category demotion (e.g. non-coding skills while pairing
# on code). Demoted categories stay in the index as a single names-only
# line — descriptions are dropped to cut noise, but every skill name
# remains visible so memory-anchored recall ("load <name>") keeps working.
# NEVER remove entries entirely: agent-created skills are the model's
# project memory, and models don't reach for skills_list to rediscover
# what the index stops showing them. Match on the top-level category
# segment so nested categories ("social-media/twitter") are demoted with
# their parent.
demoted = frozenset(
cat for cat in skills_by_category
if cat.split("/", 1)[0] in (compact_categories or frozenset())
)
hidden_note = ""
if hidden_categories:
before = sum(len(v) for v in skills_by_category.values())
skills_by_category = {
cat: entries
for cat, entries in skills_by_category.items()
if cat.split("/", 1)[0] not in hidden_categories
}
pruned = before - sum(len(v) for v in skills_by_category.values())
if pruned:
hidden_note = (
f"\n(Note: {pruned} skill(s) in categories unrelated to the "
"current coding context are not listed here. The full catalog "
"is available via skills_list if the user asks for something "
"outside this list.)"
)
if demoted:
hidden_note = (
"\n(Categories marked [names only] are outside the current coding "
"context, so their descriptions are omitted — the skills work "
"normally and load with skill_view(name) as usual.)"
)
if not skills_by_category:
result = ""
else:
index_lines = []
for category in sorted(skills_by_category.keys()):
# Deduplicate and sort skills within each category
seen = set()
if category in demoted:
names = sorted({name for name, _ in skills_by_category[category]})
index_lines.append(f" {category} [names only]: {', '.join(names)}")
continue
cat_desc = category_descriptions.get(category, "")
if cat_desc:
index_lines.append(f" {category}: {cat_desc}")
else:
index_lines.append(f" {category}:")
# Deduplicate and sort skills within each category
seen = set()
for name, desc in sorted(skills_by_category[category], key=lambda x: x[0]):
if name in seen:
continue

15
agent/system_prompt.py
View file

@ -191,21 +191,22 @@ def build_system_prompt_parts(agent: Any, system_message: Optional[str] = None)
)
if toolset
}
# Coding posture prunes non-coding skill categories from the index
# (discovery-only — skills_list/skill_view still reach everything).
_hidden_cats = frozenset()
# Coding posture demotes non-coding skill categories to names-only in
# the index (never hidden — skill_view/skills_list reach everything,
# and every name stays visible for memory-anchored recall).
_compact_cats = frozenset()
try:
from agent.coding_context import coding_hidden_skill_categories
from agent.coding_context import coding_compact_skill_categories
_hidden_cats = coding_hidden_skill_categories(
_compact_cats = coding_compact_skill_categories(
platform=agent.platform, cwd=resolve_context_cwd()
)
except Exception:
_hidden_cats = frozenset()
_compact_cats = frozenset()
skills_prompt = _r.build_skills_system_prompt(
available_tools=agent.valid_tool_names,
available_toolsets=avail_toolsets,
hidden_categories=_hidden_cats or None,
compact_categories=_compact_cats or None,
)
else:
skills_prompt = ""

30
tests/agent/test_coding_context.py
View file

@ -368,20 +368,24 @@ class TestProfiles:
assert cc.GENERAL_PROFILE.toolset is None
assert cc.GENERAL_PROFILE.guidance == ""
def test_skill_pruning_scoped_to_coding_posture(self, tmp_path):
# Coding posture hides clearly-non-coding categories; coding-adjacent
# ones stay visible (deny-list semantics).
def test_skill_demotion_scoped_to_coding_posture(self, tmp_path):
# Coding posture demotes clearly-non-coding categories to names-only
# in the index (never hides them — agent-created skills are the
# model's project memory and must stay recallable by name).
# Coding-adjacent categories keep full entries (deny-list semantics).
_git_init(tmp_path)
coding = cc.resolve_runtime_mode(platform="cli", cwd=tmp_path, config={})
hidden = coding.hidden_skill_categories()
assert "social-media" in hidden and "smart-home" in hidden
for kept in ("github", "devops", "software-development", "data-science"):
assert kept not in hidden
# General posture hides nothing.
general = cc.resolve_runtime_mode(
platform="telegram", cwd=tmp_path, config={}
)
assert general.hidden_skill_categories() == frozenset()
for raw in ("auto", "on", "focus"):
mode = cc.resolve_runtime_mode(
platform="cli", cwd=tmp_path, config={"agent": {"coding_context": raw}}
)
assert mode.is_coding is True
compact = mode.compact_skill_categories()
assert "social-media" in compact and "smart-home" in compact
for kept in ("github", "devops", "software-development", "data-science"):
assert kept not in compact
# General posture demotes nothing.
general = cc.resolve_runtime_mode(platform="telegram", cwd=tmp_path, config={})
assert general.compact_skill_categories() == frozenset()
# ── detection signals ───────────────────────────────────────────────────────

40
tests/agent/test_prompt_builder.py
View file

@ -276,8 +276,14 @@ class TestBuildSkillsSystemPrompt:
# "search" should appear only once per category
assert result.count("- search") == 1
def test_hidden_categories_pruned_with_note(self, monkeypatch, tmp_path):
"""Posture-driven pruning drops whole categories and discloses it."""
def test_compact_categories_demoted_to_names_only(self, monkeypatch, tmp_path):
"""Posture-driven demotion keeps every skill NAME visible.
Demoted categories lose their descriptions, never their entries —
full pruning caused silent capability loss in a real workflow
(agent-created skills are the model's project memory, and models
don't rediscover them via skills_list once the index goes quiet).
"""
monkeypatch.setenv("HERMES_HOME", str(tmp_path))
for cat, name in (("social-media", "tweet-stuff"), ("github", "pr-review")):
d = tmp_path / "skills" / cat / name
@ -287,14 +293,18 @@ class TestBuildSkillsSystemPrompt:
)
result = build_skills_system_prompt(
hidden_categories=frozenset({"social-media"})
compact_categories=frozenset({"social-media"})
)
assert "pr-review" in result
assert "tweet-stuff" not in result
# Disclosure note so the model knows the full catalog exists.
assert "skills_list" in result
# Coding-adjacent category keeps its full entry.
assert "pr-review" in result and "Does pr-review things" in result
# Demoted category: name stays visible, description is dropped.
assert "tweet-stuff" in result
assert "Does tweet-stuff things" not in result
assert "social-media [names only]" in result
# Disclosure note explains the demotion and how to load.
assert "skill_view" in result
def test_hidden_categories_prune_nested_and_miss_cache_separately(
def test_compact_categories_demote_nested_and_miss_cache_separately(
self, monkeypatch, tmp_path
):
monkeypatch.setenv("HERMES_HOME", str(tmp_path))
@ -303,14 +313,16 @@ class TestBuildSkillsSystemPrompt:
(d / "SKILL.md").write_text(
"---\nname: thread-writer\ndescription: Write threads\n---\n"
)
# Nested category ("social-media/twitter") pruned via its parent.
pruned = build_skills_system_prompt(
hidden_categories=frozenset({"social-media"})
# Nested category ("social-media/twitter") demoted via its parent:
# name visible, description gone.
compact = build_skills_system_prompt(
compact_categories=frozenset({"social-media"})
)
assert "thread-writer" not in pruned
# Unfiltered call must not be served from the filtered cache entry.
assert "thread-writer" in compact
assert "Write threads" not in compact
# Unfiltered call must not be served from the compacted cache entry.
full = build_skills_system_prompt()
assert "thread-writer" in full
assert "Write threads" in full
def test_excludes_incompatible_platform_skills(self, monkeypatch, tmp_path):
"""Skills with platforms: [macos] should not appear on Linux."""