* feat(skills): add osint-investigation optional skill (closes#355)
Phase-1 public-records OSINT investigation framework adapted from
ShinMegamiBoson/OpenPlanter (MIT). Lives in optional-skills/research/.
Six data-source wiki entries (FEC, SEC EDGAR, USAspending, Senate LD,
OFAC SDN, ICIJ Offshore Leaks), each following the 9-section template:
summary, access, schema, coverage, cross-reference keys, data quality,
acquisition, legal, references.
Six stdlib-only acquisition scripts that emit normalized CSV, plus three
analysis scripts:
- entity_resolution.py — three-tier match (exact / fuzzy / token overlap)
with explicit confidence per row
- timing_analysis.py — permutation test for donation/contract timing
correlation, joins through cross-links
- build_findings.py — assembles structured findings.json with
evidence chains pointing back to source rows
Validation: full pipeline runs end-to-end on synthetic fixtures. Entity
resolution found 24 cross-matches with 0 false positives on a 5-row /
4-row test set. Timing analysis on 5 donations clustered near 3 awards
returned p=0.000, effect size 2.41 SD. Findings JSON correctly tags
HIGH-severity timing pattern. All 9 scripts pass --help and py_compile.
Docs site page auto-generated by website/scripts/generate-skill-docs.py;
sidebar + catalog entries updated by the same generator.
* fix(osint-investigation): live API fixes from end-to-end sweep
Live-tested the skill on a real public-citizen query and found three bugs
the synthetic E2E missed. All three are now fixed and re-verified.
1. FEC fetch hung on contributor name searches.
The combination of two_year_transaction_period + sort=date +
contributor_name puts the OpenFEC query plan on a slow path that the
upstream gateway times out (25s+). Switched to min_date/max_date with no
explicit sort. Renamed --candidate to --contributor (the original name
was misleading: FEC searches by donor, not by candidate; --candidate is
kept as a deprecated alias). Added --state filter for narrowing.
2. ICIJ Offshore Leaks reconcile endpoint returns 404.
ICIJ removed the Open Refine reconciliation API. Rewrote
fetch_icij_offshore.py to download the official bulk CSV ZIP (~70 MB,
public, no auth) and search it locally. Cached under
$HERMES_OSINT_CACHE/icij/ (default ~/.cache/hermes-osint/icij/) for
30 days, --force-refresh to refetch. Verified live: 'PUTIN' query
returns 5 Panama Papers officer matches in 0.5s after first download.
3. SEC EDGAR silently returned 0 when the company-name resolver matched
an individual Form 3/4/5 filer (insider trading disclosures).
Now surfaces 'Resolved company X → CIK Y (Z)' on stderr, prints a
filing-type histogram when the type filter wipes results, and
explicitly warns when the matched CIK appears to be an individual
filer rather than a corporate registrant.
Bonus: _http.py was retrying 429 responses with exponential backoff plus
honoring (often-missing) Retry-After headers, which compounded into
multi-second hangs per page when the upstream key was over quota.
Changed to fail-fast on 429 with a clear, actionable error showing the
upstream's quota message. Verified: 0.3s fast-fail vs the previous 60s
hang on DEMO_KEY rate-limit exhaustion.
Updated SKILL.md, fec.md, and icij-offshore.md to match the new CLI
flags and ICIJ bulk-cache flow. Regenerated the docusaurus page via
website/scripts/generate-skill-docs.py.
Live sweep results across all 6 sources for 'Dillon Rolnick, New York':
- OFAC SDN: 0 matches ✓ (correctly not sanctioned)
- USAspending: 0 matches ✓ (correctly not a federal contractor)
- Senate LDA: 0 matches ✓ (correctly not a lobbying client)
- SEC EDGAR: warns it resolved to 'Rolnick Michael' (CIK 0001845264)
who is an individual Form 3 filer, not a corporate registrant
- ICIJ: 0 matches ✓ (correctly not in any offshore leak)
- FEC: rate-limited (DEMO_KEY); fails fast with clear quota message
* feat(osint-investigation): expand to 12 sources covering identity, property, courts, archives, news
Phase-2 expansion per Teknium feedback that the original 6-source skill
(federal financial/regulatory only) wasn't a complete OSINT toolkit. Adds
6 more sources covering the major omissions a real investigation would
reach for first.
New sources (6 fetch scripts + 6 wiki entries):
1. NYC ACRIS — Real property records (deeds, mortgages, liens) via the
city's Socrata API. Search by party name or property address. Joins
Parties to Master to populate doc_type, dates, borough, and amount.
Coverage: 5 NYC boroughs, ~70M party records, 1966-present.
2. OpenCorporates — Global corporate registry covering 130+ jurisdictions
(~200M companies). Free API token at
https://opencorporates.com/api_accounts/new raises the rate limit;
HTML fallback works without one (limited fields).
3. CourtListener (Free Law Project) — federal + state court opinions
(~10M back to colonial era) + PACER dockets via RECAP. Anonymous v4
search works; COURTLISTENER_TOKEN raises rate limits.
4. Wayback Machine CDX — historical web captures (~900B+). Used both for
surveillance-of-record (when did this site change?) and as a
content-recovery layer when other sources point to dead URLs.
5. Wikipedia + Wikidata — narrative bio + structured facts. Wikipedia
OpenSearch for article matching, REST summary for extracts, Wikidata
Action API (wbgetentities) for claims. Avoids the SPARQL Query
Service which is aggressively rate-limited.
6. GDELT 2.0 DOC API — global news monitoring in 100+ languages,
~2015-present. Auto-retries with 6s backoff on the standard
1-req-per-5-sec throttle.
Other changes in this commit:
- SEC EDGAR no longer raises SystemExit when the company-name resolver
finds no CIK; writes an empty CSV with header so the rest of a
pipeline can keep moving and the warning is just on stderr.
- _http.py User-Agent updated per Wikimedia policy: includes app name,
version, and a 'set HERMES_OSINT_UA to identify yourself' instruction.
- SKILL.md workflow now groups sources into two clusters (federal
financial vs identity/property/courts/archives/news) with bash
examples for each. 'When to use this skill' lists the broader set of
investigation patterns the expanded sources unlock.
Live sweep results on 'Dillon Rolnick, New York' across all 12 sources:
ofac ✓ 0 (correctly clean)
icij ✓ 0 (correctly not in any leak)
usaspending ✓ 0 (correctly not a federal contractor)
senate_lda ✓ 0 (correctly not a lobbying client)
sec_edgar ✓ 0, warns: resolved to 'Rolnick Michael' (CIK 0001845264),
individual Form 3 filer, NOT a corporate registrant
fec — rate-limited (DEMO_KEY exhausted), fails fast with
clear quota message
nyc_acris ✓ 200 records named Rolnick across NYC; 48 records at
571 Hudson (the property the web identifies as his)
opencorporates ✓ 0 (no API token configured; HTML fallback)
courtlistener ✓ 0 for 'Dillon Rolnick'; 20 for 'Rolnick' generally;
5 for 'Microsoft' sanity check
wayback ✓ 30 captures of nousresearch.com from 2011-present
wikipedia ✓ 0 (correctly not notable enough); Bill Gates sanity
returns full structured facts (occupation, employer,
DOB, place of birth, country)
gdelt ✓ 0 for 'Dillon Rolnick'; 5 for 'Nous Research'
All 17 scripts compile clean and pass --help. Synthetic analysis pipeline
regression still passes (entity_resolution 30 matches, timing p=0.000,
findings 2).
* feat(osint-investigation): remove FEC; DEMO_KEY rate-limits make it unreliable
The FEC fetcher consistently failed the live sweep because the OpenFEC
DEMO_KEY tier (40 calls/hour) exhausts on a single investigation, and
the upstream returns slow-path query plans for unindexed contributor-name
searches that the gateway times out. Without a real API key it's not
usable; with one the user has to sign up at api.data.gov first. That's
too much setup friction for a skill that should work out of the box.
Removed:
- scripts/fetch_fec.py
- references/sources/fec.md
Updated:
- SKILL.md frontmatter description + tags
- 'When NOT to use' now points users at https://www.fec.gov/data/ for
federal donations
- entity_resolution example switched from donor↔contractor to
lobbying-client↔contractor (Senate LDA + USAspending pair)
- timing_analysis example switched to lobbying-filings vs awards
- 8 wiki entries had their 'FEC ↔ ...' cross-reference bullets removed
11 sources remain (5 federal financial + 6 identity/property/courts/
archives/news). All scripts compile, pass --help, and the synthetic
analysis pipeline still passes on the new lobbying-shaped regression
fixture (30 matches, p=0.000 on tight clustering, 2 findings).
Thin wrapper around Imbue's darwinian_evolver (AGPL-3.0, subprocess-only).
Ships a working OpenRouter driver (parrot_openrouter.py), a snapshot
inspector (show_snapshot.py), and a custom-problem template. SKILL.md
has 58-char description, Pitfalls sourced from actually running the loop:
non-viable seed trap, Azure content filter killing runs, loop.run() being
a generator, nested-pickle snapshots, and aggressive default concurrency.
Salvaged from #12719 by @Bihruze — original PR shipped 12,289 LOC across
61 files (29 Python modules, FastAPI dashboard, VS Code extension,
benchmark hub, marketplace, etc.) which was far beyond the scope of the
underlying issue (#336). This version stays at the ~700-LOC scope that
issue actually asked for. Authorship of the original effort credited via
AUTHOR_MAP entry and the SKILL.md author field.
Verified end-to-end: seed 'Say {{ phrase }}' (score 0.000) evolved into
'Please repeat the following phrase exactly as it is, without any
modifications or additional formatting: {{ phrase }}' (score 0.750)
across 3 iterations on gpt-4o-mini via OpenRouter.
Co-authored-by: Bihruze <98262967+Bihruze@users.noreply.github.com>
The platforms-frontmatter sweep inserted 'platforms: [linux, macos, windows]'
immediately after 'description: >' on 5 optional-skills, landing inside the
folded scalar and breaking YAML parsing. docs-site-checks tripped on
one-three-one-rule/SKILL.md and would have failed on the other 4 in turn.
Fixed files:
- optional-skills/communication/one-three-one-rule/SKILL.md
- optional-skills/health/fitness-nutrition/SKILL.md
- optional-skills/health/neuroskill-bci/SKILL.md
- optional-skills/research/drug-discovery/SKILL.md
- optional-skills/security/oss-forensics/SKILL.md
Moved each platforms line below the closing of the description block.
All 161 SKILL.md files across the repo now parse as valid YAML.
Extends the Windows-gating work to the optional-skills/ tree. Every
SKILL.md that previously omitted the platforms: field now carries an
explicit declaration, which Hermes's loader (agent.skill_utils.
skill_matches_platform) honors to skip-load on incompatible OSes.
58 skills declared cross-platform (platforms: [linux, macos, windows]):
autonomous-ai-agents/blackbox, autonomous-ai-agents/honcho
blockchain/base, blockchain/solana
communication/one-three-one-rule
creative/blender-mcp, creative/concept-diagrams, creative/hyperframes,
creative/kanban-video-orchestrator, creative/meme-generation
devops/cli (inference-sh-cli), devops/docker-management
dogfood/adversarial-ux-test
email/agentmail
finance/3-statement-model, finance/comps-analysis, finance/dcf-model,
finance/excel-author, finance/lbo-model, finance/merger-model,
finance/pptx-author
health/fitness-nutrition, health/neuroskill-bci
mcp/fastmcp, mcp/mcporter
migration/openclaw-migration
mlops/accelerate, mlops/chroma, mlops/clip, mlops/guidance,
mlops/hermes-atropos-environments, mlops/huggingface-tokenizers,
mlops/instructor, mlops/lambda-labs, mlops/llava, mlops/modal,
mlops/peft, mlops/pinecone, mlops/pytorch-lightning, mlops/qdrant,
mlops/saelens, mlops/simpo, mlops/stable-diffusion
productivity/canvas, productivity/shop-app, productivity/shopify,
productivity/siyuan, productivity/telephony
research/domain-intel, research/drug-discovery, research/duckduckgo-search,
research/gitnexus-explorer, research/parallel-cli, research/scrapling
security/1password, security/oss-forensics, security/sherlock
web-development/page-agent
5 skills gated from Windows (platforms: [linux, macos]):
mlops/flash-attention - Flash Attention wheels are Linux-first; Windows
install requires building from source with CUDA
mlops/faiss - faiss-gpu has no Windows wheel; gate rather than
leak partial (faiss-cpu) support
mlops/nemo-curator - NVIDIA NeMo ecosystem has no first-class Windows path
mlops/slime - Megatron+SGLang RL stack is Linux-only in practice
mlops/whisper - openai-whisper + ffmpeg setup on Windows is
non-trivial; gate until Windows install stanza lands
Methodology: scanned every SKILL.md for Windows-hostile signals
(apt-get, brew, systemd, osascript, ptrace, X11 binaries, POSIX-only
Python APIs, Docker POSIX $(pwd) bind-mounts, explicit 'linux-only' /
'macos-only' text). 3 skills flagged as having hard signals on review:
docker-management and qdrant only had POSIX $(pwd) docker examples and
the tools themselves (Docker Desktop, Qdrant) run fine on Windows —
declared ALL. whisper had an apt/brew ffmpeg install path and nothing
else but the openai-whisper Windows install story is rough enough to
warrant gating.
Strict-over-lenient policy: when in doubt, gate. Easier to un-gate after
verified Windows support lands than to leak partial support that
manifests as mid-task failures for Windows users.
Hermes's skill loader (agent/skill_utils.skill_matches_platform) already honors
the 'platforms:' frontmatter field and skip-loads skills whose declared
platform list doesn't include sys.platform. Seven bundled skills are in fact
Linux/macOS-only but never declared it, so they leak into Windows skill
listings and sometimes load with broken instructions.
Audited all 160 SKILL.md files (skills/ + optional-skills/) for Windows-
hostile signals: apt-get/brew/systemd/chmod+x install flows, ptrace/proc
runtime dependencies, bash-only launcher scripts, and package dependencies
with no Windows build. The 7 below fail one or more of those tests in a way
that fundamentally can't be papered over by docs edits:
minecraft-modpack-server bash start.sh + chmod +x + apt openjdk
evaluating-llms-harness lm-eval-harness bash launcher scripts
distributed-llm-pretraining-
torchtitan bash multi-node torchrun launcher
python-debugpy remote attach relies on /proc ptrace_scope
pytorch-fsdp NCCL backend; Windows path is WSL only
tensorrt-llm NVIDIA TensorRT-LLM has no Windows build
searxng-search Docker volume flow assumes POSIX $(pwd)
All seven get 'platforms: [linux, macos]'. On Windows the loader now skips
them silently — no more phantom skill listings, no more mid-task failures
because an Apple-only path was surfaced as a suggestion.
Cross-platform skills that merely CONTAIN signals in examples or
install-instructions (brew install as one of several paths, /tmp/ in a code
snippet, etc.) are NOT touched by this commit. A broader audit that
declares the ~140 cross-platform skills as 'platforms: [linux, macos,
windows]' can follow as a separate change once each has been verified
working on Windows.
The installed user copies under ~/AppData/Local/hermes/skills/ (when they
exist) are also patched so the running session reflects the gating
immediately, but only the in-repo files are committed here.
Closes the remaining gaps from PR #11562 that weren't covered by the
core SearXNG integration landed in #20823.
- optional-skills/research/searxng-search/ — installable skill with
SKILL.md (curl-based usage, category support, Python example) and
searxng.sh helper script for health checks and instance queries
- website/docs/user-guide/configuration.md — SearXNG added to the
Web Search Backends section (5 backends, backend table, per-capability
split config example, correct search-only note)
- website/docs/reference/environment-variables.md — SEARXNG_URL row
- website/docs/reference/optional-skills-catalog.md — searxng-search entry
The core SearXNG code, OPTIONAL_ENV_VARS, hermes tools picker, and tests
were already on main via #20823. This commit is purely additive docs +
the optional skill scaffold.
Credits from #11562 salvage:
@w4rum — original _searxng_search structure
@nathansdev — tools_config.py integration
@moyomartin — category support and result formatting
@0xMihai — config/env var approach
@nicobailon — skill and documentation structure
@searxng-fan — error handling patterns
@local-first — self-hosted-first philosophy and docs
Index codebases with GitNexus and serve an interactive knowledge
graph web UI via Cloudflare tunnel. No sudo required.
Includes:
- Full setup/build/serve/tunnel pipeline
- Zero-dependency Node.js reverse proxy script
- Pitfalls section covering cloudflared config conflicts,
Vite allowedHosts, Claude Code artifact cleanup, and
browser memory limits for large repos
* feat(gateway): skill-aware slash commands, paginated /commands, Telegram 100-cap
Map active skills to Telegram's slash command menu so users can
discover and invoke skills directly. Three changes:
1. Telegram menu now includes active skill commands alongside built-in
commands, capped at 100 entries (Telegram Bot API limit). Overflow
commands remain callable but hidden from the picker. Logged at
startup when cap is hit.
2. New /commands [page] gateway command for paginated browsing of all
commands + skills. /help now shows first 10 skill commands and
points to /commands for the full list.
3. When a user types a slash command that matches a disabled or
uninstalled skill, they get actionable guidance:
- Disabled: 'Enable it with: hermes skills config'
- Optional (not installed): 'Install with: hermes skills install official/<path>'
Built on ideas from PR #3921 by @kshitijk4poor.
* chore: move 21 niche skills to optional-skills
Move specialized/niche skills from built-in (skills/) to optional
(optional-skills/) to reduce the default skill count. Users can
install them with: hermes skills install official/<category>/<name>
Moved skills (21):
- mlops: accelerate, chroma, faiss, flash-attention,
hermes-atropos-environments, huggingface-tokenizers, instructor,
lambda-labs, llava, nemo-curator, pinecone, pytorch-lightning,
qdrant, saelens, simpo, slime, tensorrt-llm, torchtitan
- research: domain-intel, duckduckgo-search
- devops: inference-sh cli
Built-in skills: 96 → 75
Optional skills: 22 → 43
* fix: only include repo built-in skills in Telegram menu, not user-installed
User-installed skills (from hub or manually added) stay accessible via
/skills and by typing the command directly, but don't get registered
in the Telegram slash command picker. Only skills whose SKILL.md is
under the repo's skills/ directory are included in the menu.
This keeps the Telegram menu focused on the curated built-in set while
user-installed skills remain discoverable through /skills and /commands.
Add two new optional skills:
- siyuan (optional-skills/productivity/): SiYuan Note knowledge base
API skill — search, read, create, and manage blocks/documents in a
self-hosted SiYuan instance via curl. Requires SIYUAN_TOKEN.
- scrapling (optional-skills/research/): Intelligent web scraping skill
using the Scrapling library — anti-bot fetching, Cloudflare bypass,
CSS/XPath selectors, spider framework for multi-page crawling.
Placed in optional-skills/ (not bundled) since both are niche tools
that require external dependencies.
Co-authored-by: FEUAZUR <FEUAZUR@users.noreply.github.com>
parallel-cli is a paid third-party vendor skill that requires
PARALLEL_API_KEY, but it was shipped in the default skills/ directory
with no env-var gate. This caused it to appear in every user's system
prompt even when they have no Parallel account or API key.
Move it to optional-skills/ so it is only visible through the Skills
Hub and must be explicitly installed. Also remove it from the default
skills catalog docs.
Meta-skill that indexes 400+ bioinformatics skills from two open-source
repos (GPTomics/bioSkills and ClawBio/ClawBio) and fetches domain-specific
reference material on demand. Covers genomics, transcriptomics, single-cell,
variant calling, pharmacogenomics, metagenomics, structural biology, and
20+ other computational biology domains.
No dependencies bundled — the skill clones the relevant repo when needed
and reads the domain-specific guides as reference material.
Add official optional skill for qmd (tobi/qmd), a local on-device
search engine for personal knowledge bases, notes, docs, and meeting
transcripts.
Covers:
- Installation and setup for macOS and Linux
- Collection management and context annotations
- All search modes: BM25, vector, hybrid with reranking
- MCP integration (stdio and HTTP daemon modes)
- Structured query patterns and best practices
- systemd/launchd service configs for daemon persistence
Placed in optional-skills/ due to heavyweight requirements
(Node >= 22, ~2GB local models).
