mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-06-27 11:22:03 +00:00
fix(learn): teach /learn the full CONTRIBUTING.md skill standards (#52372)
The /learn authoring prompt taught a subset of the HARDLINE skill rules, and stated the <=60-char description rule without making the model enforce it — so generated descriptions overshot (up to 202 chars), which the 60-char system-prompt skill index then silently truncates. - description: add the index-truncation rationale, a count-and-trim self-check, and a good/bad length example so the model actually hits <=60. - add platforms-gating rule (OS-bound primitives -> declare platforms:). - add author-credits-human-first rule. - round out the Hermes-tool framing with the full wrapped-tool mapping and references/templates layout. Closes #52367.
This commit is contained in:
parent
60a2feeebf
commit
e62afaca62
2 changed files with 50 additions and 9 deletions
|
|
@ -28,15 +28,32 @@ from __future__ import annotations
|
|||
# (HARDLINE)" and the hermes-agent-dev new-skill salvage reference. Embedded in
|
||||
# the prompt so the agent authors skills the way a maintainer would by hand.
|
||||
_AUTHORING_STANDARDS = """\
|
||||
Follow the Hermes skill-authoring standards exactly:
|
||||
Follow the Hermes skill-authoring standards exactly. These are the same
|
||||
HARDLINE rules a maintainer enforces in review:
|
||||
|
||||
Frontmatter:
|
||||
- name: lowercase-hyphenated, <=64 chars, no spaces.
|
||||
- description: ONE sentence, <=60 characters, ends with a period. State the
|
||||
- description: ONE sentence, **<=60 characters**, ends with a period. State the
|
||||
capability, not the implementation. No marketing words (powerful,
|
||||
comprehensive, seamless, advanced). Do NOT repeat the skill name. If the
|
||||
description contains a colon, wrap the whole value in double quotes.
|
||||
comprehensive, seamless, advanced, robust). Do NOT repeat the skill name. If
|
||||
the description contains a colon, wrap the whole value in double quotes.
|
||||
This is the most-violated rule and it is NOT cosmetic: the system-prompt
|
||||
skill index truncates the description to 60 chars and loads it every
|
||||
session, so anything past char 60 is silently cut and never routes. After
|
||||
you write the description, COUNT the characters; if it is over 60, cut it
|
||||
down before saving — do not ship a sentence and hope.
|
||||
Good (<=60): `Search arXiv papers by keyword, author, or ID.`
|
||||
Bad (123): `A comprehensive skill that lets the agent search arXiv for
|
||||
academic papers using keywords, authors, and categories.`
|
||||
- version: 0.1.0
|
||||
- author: the human you are authoring this for, first; "Hermes Agent" second.
|
||||
Never credit only the tool.
|
||||
- platforms: declare `[macos]`, `[linux]`, and/or `[windows]` IF the skill
|
||||
uses OS-bound primitives (osascript/apt/systemctl => the matching OS; /proc,
|
||||
os.setsid, signal.SIGKILL => linux; fcntl/termios => POSIX). Prefer fixing it
|
||||
cross-platform first (tempfile.gettempdir(), pathlib.Path, psutil); gate only
|
||||
when the dependency is genuinely platform-bound. Omit the field for portable
|
||||
skills.
|
||||
- metadata.hermes.tags: a few Capitalized, Relevant, Tags.
|
||||
|
||||
Body section order (omit a section only if it genuinely has no content):
|
||||
|
|
@ -52,10 +69,16 @@ Body section order (omit a section only if it genuinely has no content):
|
|||
|
||||
Hermes-tool framing (this is what makes it a skill, not shell docs):
|
||||
- Frame running scripts as "invoke through the `terminal` tool".
|
||||
- Use `read_file` (not cat/head/tail), `search_files` (not grep/find/ls),
|
||||
`patch` (not sed/awk), `web_extract` (not curl-to-scrape),
|
||||
`vision_analyze` for images. Reference these tools by name in backticks.
|
||||
- Do NOT name shell utilities the agent already has wrapped.
|
||||
- Reference Hermes tools by name in backticks: `terminal`, `read_file`,
|
||||
`write_file`, `search_files`, `patch`, `web_extract`, `web_search`,
|
||||
`vision_analyze`, `browser_navigate`, `delegate_task`, `image_generate`,
|
||||
`text_to_speech`, `cronjob`, `memory`, `skill_view`, `execute_code`.
|
||||
- Do NOT name shell utilities the agent already has wrapped: say `read_file`
|
||||
not cat/head/tail, `search_files` not grep/rg/find/ls, `patch` not sed/awk,
|
||||
`web_extract` not curl-to-scrape, `write_file` not echo>file or heredocs.
|
||||
- Third-party CLIs (ffmpeg, gh, an SDK) are fine inside a script file, but the
|
||||
prose still frames them as "invoke through the `terminal` tool". If the
|
||||
skill needs an MCP server, name it and document its setup in Prerequisites.
|
||||
|
||||
Quality bar:
|
||||
- Prefer exact commands, endpoint URLs, function signatures, and config keys
|
||||
|
|
@ -66,7 +89,8 @@ Quality bar:
|
|||
- Don't write a router/index/hub skill that only points at other skills.
|
||||
- Larger scripts/parsers belong in a `scripts/` file (add via
|
||||
`skill_manage` write_file), referenced from SKILL.md by relative path — not
|
||||
inlined for the agent to re-type every run."""
|
||||
inlined for the agent to re-type every run. References go in `references/`,
|
||||
templates in `templates/`."""
|
||||
|
||||
|
||||
def build_learn_prompt(user_request: str) -> str:
|
||||
|
|
|
|||
|
|
@ -46,6 +46,23 @@ class TestBuildLearnPrompt:
|
|||
# The single most-violated rule must be explicit in the prompt.
|
||||
assert "60" in _AUTHORING_STANDARDS
|
||||
|
||||
def test_teaches_the_full_hardline_standards(self):
|
||||
# /learn must teach ALL the CONTRIBUTING.md skill rules, not just the
|
||||
# description length — otherwise distilled skills miss platform gating,
|
||||
# author credit, and the tool-framing table. Lock the coverage in.
|
||||
std = _AUTHORING_STANDARDS.lower()
|
||||
# #1 description: the count-and-trim self-check (the reported bug).
|
||||
assert "count" in std and "60" in std
|
||||
# #3 platforms gating against OS-bound primitives.
|
||||
assert "platforms" in std
|
||||
# #4 author credits the human first.
|
||||
assert "author" in std
|
||||
# #2 Hermes-tool framing names the wrapped tools, not shell utilities.
|
||||
for tool in ("read_file", "search_files", "patch", "write_file"):
|
||||
assert tool in std
|
||||
# #6 scripts/references/templates layout.
|
||||
assert "scripts/" in _AUTHORING_STANDARDS
|
||||
|
||||
|
||||
class TestLearnRegistryWiring:
|
||||
def test_learn_is_registered_and_resolves(self):
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue