mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-05-04 02:21:47 +00:00
fa9383d27b
Based on three live test runs against 346 agent-created skills on the author's own setup (~6.5 min, opus-4.7, 86 API calls), the curator prompt needed three sharpenings before it consistently produced real umbrella consolidation instead of passive audit output: **Umbrella-first framing.** The original 'decide keep/patch/archive/ consolidate' framing lets opus default to 'keep' whenever two skills aren't byte-identical. The new prompt explicitly tells the reviewer that pairwise distinctness is the wrong bar — the right question is 'would a human maintainer write this as N separate skills, or one skill with N labeled subsections?' Expect 10-25 prefix clusters; merge each into an umbrella via one of three methods. **Three concrete consolidation methods.** (a) Merge into an existing umbrella (patch the broadest skill, archive siblings); (b) Create a new umbrella SKILL.md (skill_manage action=create); (c) Demote session-specific detail into references/, templates/, or scripts/ under the umbrella via skill_manage action=write_file, then archive the narrow sibling. This matches the support-file vocabulary the review-prompt side already uses (PR #17213). **Two observed bailouts pre-empted:** 'usage counters are zero so I can't judge' (rule 4: judge on content, not use_count) and 'each has a distinct trigger' (rule 5: pairwise distinctness is the wrong bar). **Config-aware parent inheritance.** _run_llm_review() was building AIAgent() without explicit provider/model, hitting an auto-resolve path that returned empty credentials → HTTP 400 'No models provided' against OpenRouter. Fork now inherits the user's main provider and model (via load_config + resolve_runtime_provider) before spawning — runs on whatever the user is currently on, OAuth-backed or pool-backed included. **Unbounded iteration ceiling.** max_iterations=8 was way too low for an umbrella-build pass over hundreds of skills. A live pass takes 50-100 API calls (scanning, clustering, skill_view'ing candidates, patching umbrellas, mv'ing siblings). Raised to 9999 — the natural stopping criterion is 'no more clusters worth processing', not an arbitrary tool-call budget. **Tests updated:** test_curator_review_prompt_has_invariants accepts DO NOT / MUST NOT and drops 'keep' from the required-verb set (the umbrella-first prompt correctly deemphasizes 'keep' as a first-class decision label since passive keep-everything is the failure mode being prevented). Added test_curator_review_prompt_is_umbrella_first asserting the umbrella framing, class-level thinking, references/ + templates/ + scripts/ support-file mentions, and the 'use_count is not evidence of value' pre-emption. Added test_curator_review_prompt_offers_support_file_actions asserting skill_manage action=create and action=write_file are both named. **Live validation on author's setup:** - Run 1 (old prompt): 3 archives, stopped after surveying — typical passive outcome - Run 2 (consolidation prompt): 44 archives, 3 patches, surfaced the 50-skill mlops reorg duplicate bug but didn't umbrella - Run 3 (this prompt): 249 archives + 18 new class-level umbrellas created, reducing agent-created skills from 346 → 118 with every archived skill's content preserved as references/ under its umbrella. Pinned skill untouched. Full report in PR description. |
||
|---|---|---|
| .. | ||
| transports | ||
| __init__.py | ||
| test_anthropic_adapter.py | ||
| test_anthropic_keychain.py | ||
| test_auxiliary_client.py | ||
| test_auxiliary_client_anthropic_custom.py | ||
| test_auxiliary_config_bridge.py | ||
| test_auxiliary_main_first.py | ||
| test_auxiliary_named_custom_providers.py | ||
| test_auxiliary_transport_autodetect.py | ||
| test_bedrock_1m_context.py | ||
| test_bedrock_adapter.py | ||
| test_bedrock_integration.py | ||
| test_codex_cloudflare_headers.py | ||
| test_compress_focus.py | ||
| test_compressor_image_tokens.py | ||
| test_context_compressor.py | ||
| test_context_engine.py | ||
| test_context_references.py | ||
| test_copilot_acp_client.py | ||
| test_credential_pool.py | ||
| test_credential_pool_routing.py | ||
| test_crossloop_client_cache.py | ||
| test_curator.py | ||
| test_direct_provider_url_detection.py | ||
| test_display.py | ||
| test_display_emoji.py | ||
| test_error_classifier.py | ||
| test_external_skills.py | ||
| test_gemini_cloudcode.py | ||
| test_gemini_free_tier_gate.py | ||
| test_gemini_native_adapter.py | ||
| test_gemini_schema.py | ||
| test_image_gen_registry.py | ||
| test_image_routing.py | ||
| test_insights.py | ||
| test_kimi_coding_anthropic_thinking.py | ||
| test_local_stream_timeout.py | ||
| test_memory_provider.py | ||
| test_memory_user_id.py | ||
| test_minimax_auxiliary_url.py | ||
| test_minimax_provider.py | ||
| test_model_metadata.py | ||
| test_model_metadata_local_ctx.py | ||
| test_model_metadata_ssl.py | ||
| test_models_dev.py | ||
| test_moonshot_schema.py | ||
| test_nous_rate_guard.py | ||
| test_onboarding.py | ||
| test_prompt_builder.py | ||
| test_prompt_caching.py | ||
| test_proxy_and_url_validation.py | ||
| test_rate_limit_tracker.py | ||
| test_redact.py | ||
| test_shell_hooks.py | ||
| test_shell_hooks_consent.py | ||
| test_skill_commands.py | ||
| test_streaming_context_scrubber.py | ||
| test_subagent_progress.py | ||
| test_subagent_stop_hook.py | ||
| test_subdirectory_hints.py | ||
| test_title_generator.py | ||
| test_unsupported_parameter_retry.py | ||
| test_unsupported_temperature_retry.py | ||
| test_usage_pricing.py | ||
| test_vision_resolved_args.py | ||