mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-06-08 08:11:38 +00:00
fix(skills-hub): show every catalog source on /docs/skills (skills.sh, ClawHub, browse.sh, OpenAI, …) (#32336)
Some checks are pending
Deploy Site / deploy-vercel (push) Waiting to run
Deploy Site / deploy-docs (push) Waiting to run
Docker / shell lint / Lint Dockerfile (hadolint) (push) Waiting to run
Docker / shell lint / Lint docker/ shell scripts (shellcheck) (push) Waiting to run
Docker Build and Publish / build-amd64 (push) Waiting to run
Docker Build and Publish / build-arm64 (push) Waiting to run
Docker Build and Publish / merge (push) Blocked by required conditions
Docker Build and Publish / move-latest (push) Blocked by required conditions
Lint (ruff + ty) / ruff + ty diff (push) Waiting to run
Lint (ruff + ty) / ruff enforcement (blocking) (push) Waiting to run
Lint (ruff + ty) / Windows footguns (blocking) (push) Waiting to run
Nix / nix (macos-latest) (push) Waiting to run
Nix / nix (ubuntu-latest) (push) Waiting to run
Build Skills Index / build-index (push) Waiting to run
Build Skills Index / trigger-deploy (push) Blocked by required conditions
Tests / test (1) (push) Waiting to run
Tests / test (2) (push) Waiting to run
Tests / test (3) (push) Waiting to run
Tests / test (4) (push) Waiting to run
Tests / test (5) (push) Waiting to run
Tests / test (6) (push) Waiting to run
Tests / save-durations (push) Blocked by required conditions
Tests / e2e (push) Waiting to run
Some checks are pending
Deploy Site / deploy-vercel (push) Waiting to run
Deploy Site / deploy-docs (push) Waiting to run
Docker / shell lint / Lint Dockerfile (hadolint) (push) Waiting to run
Docker / shell lint / Lint docker/ shell scripts (shellcheck) (push) Waiting to run
Docker Build and Publish / build-amd64 (push) Waiting to run
Docker Build and Publish / build-arm64 (push) Waiting to run
Docker Build and Publish / merge (push) Blocked by required conditions
Docker Build and Publish / move-latest (push) Blocked by required conditions
Lint (ruff + ty) / ruff + ty diff (push) Waiting to run
Lint (ruff + ty) / ruff enforcement (blocking) (push) Waiting to run
Lint (ruff + ty) / Windows footguns (blocking) (push) Waiting to run
Nix / nix (macos-latest) (push) Waiting to run
Nix / nix (ubuntu-latest) (push) Waiting to run
Build Skills Index / build-index (push) Waiting to run
Build Skills Index / trigger-deploy (push) Blocked by required conditions
Tests / test (1) (push) Waiting to run
Tests / test (2) (push) Waiting to run
Tests / test (3) (push) Waiting to run
Tests / test (4) (push) Waiting to run
Tests / test (5) (push) Waiting to run
Tests / test (6) (push) Waiting to run
Tests / save-durations (push) Blocked by required conditions
Tests / e2e (push) Waiting to run
The Skills Hub page was stuck on a stale Feb 25 snapshot, showing only Built-in
+ Optional + Anthropic + LobeHub. The unified index already has 2078 skills
from skills.sh / ClawHub / LobeHub / GitHub taps / Claude Marketplace, and
BrowseShSource adds another ~330 — none of it was reaching the page.
Changes:
- website/scripts/extract-skills.py: read website/static/api/skills-index.json
(the unified multi-source catalog, rebuilt twice daily) as the canonical
external source. Keep the legacy skills/index-cache/ fallback for offline
builds. Add friendly per-source labels (skills.sh, ClawHub, browse.sh,
OpenAI, HuggingFace, Anthropic, LobeHub, etc.) and per-entry installCmd.
- website/src/pages/skills/index.tsx: add source pills + ordering for the 11
new sources; render installCmd from the index entry.
- website/scripts/prebuild.mjs: when no local skills-index.json exists, fetch
the live one from hermes-agent.nousresearch.com so local 'npm run build'
matches production without burning GitHub API quota.
- scripts/build_skills_index.py: crawl BrowseShSource so browse.sh entries
land in the unified index. Adjust source_order.
- tools/skills_hub.py: GitHubSource.DEFAULT_TAPS — openai/skills moved its
skills into skills/.curated/ and skills/.system/, so add both as explicit
taps (the listing code skips dotted dirs by design). Drop
VoltAgent/awesome-agent-skills (README-only, no SKILL.md files) and
MiniMax-AI/cli (singular skill, not a tap directory). Net effect: github
source jumps from 83 → 143 skills, with OpenAI properly included.
- .github/workflows/deploy-site.yml: build the unified index BEFORE running
extract-skills.py — previous order meant extract-skills always fell back
to the legacy cache. Drop the 'skip if file exists' guard; the file is
gitignored and must be rebuilt every deploy.
- .github/workflows/skills-index.yml: drop the broken 'deploy-with-index'
job (it cp'd 'landingpage/\*' which no longer exists, failing every cron
run since the landingpage move). Replace it with a workflow_dispatch
trigger of deploy-site.yml so the index refresh still reaches production
on schedule.
- website/docs/user-guide/features/skills.md: drop VoltAgent from the
default-taps doc list to match the code.
Before: 695 skills (Built-in 90, Optional 84, Anthropic 16, LobeHub 505).
After: 2168 skills across 9 source pills, including the 1212 skills.sh
entries the user expected to see.
This commit is contained in:
parent
c26af46811
commit
cea87d9139
8 changed files with 396 additions and 95 deletions
|
|
@ -8,6 +8,13 @@
|
|||
// CI workflows still run the extraction explicitly, which is a no-op duplicate
|
||||
// but matches their historical behaviour.
|
||||
//
|
||||
// We also try to pull a fresh copy of skills-index.json (the unified
|
||||
// multi-source catalog) from the live docs site if it's not already on disk.
|
||||
// That way local `npm run build` doesn't have to wait on
|
||||
// scripts/build_skills_index.py crawling every skill source — which takes
|
||||
// several minutes and burns GitHub API quota — but still gets the same
|
||||
// 2000+ external skills the deployed site has.
|
||||
//
|
||||
// If python3 or its deps (pyyaml) aren't available on the local machine, we
|
||||
// fall back to writing an empty skills.json so `npm run build` still
|
||||
// succeeds — the Skills Hub page just shows an empty state, and llms.txt
|
||||
|
|
@ -15,7 +22,7 @@
|
|||
// deploys get real data.
|
||||
|
||||
import { spawnSync } from "node:child_process";
|
||||
import { mkdirSync, writeFileSync, existsSync } from "node:fs";
|
||||
import { mkdirSync, writeFileSync, existsSync, statSync } from "node:fs";
|
||||
import { dirname, join, resolve } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
|
|
@ -24,6 +31,10 @@ const websiteDir = resolve(scriptDir, "..");
|
|||
const extractScript = join(scriptDir, "extract-skills.py");
|
||||
const llmsScript = join(scriptDir, "generate-llms-txt.py");
|
||||
const outputFile = join(websiteDir, "src", "data", "skills.json");
|
||||
const unifiedIndexFile = join(websiteDir, "static", "api", "skills-index.json");
|
||||
const UNIFIED_INDEX_URL =
|
||||
"https://hermes-agent.nousresearch.com/docs/api/skills-index.json";
|
||||
const UNIFIED_INDEX_MAX_AGE_MS = 24 * 60 * 60 * 1000; // 24h
|
||||
|
||||
function writeEmptyFallback(reason) {
|
||||
mkdirSync(dirname(outputFile), { recursive: true });
|
||||
|
|
@ -51,6 +62,64 @@ function runPython(script, label) {
|
|||
return true;
|
||||
}
|
||||
|
||||
async function ensureUnifiedIndex() {
|
||||
// If we have a recent copy on disk, trust it.
|
||||
if (existsSync(unifiedIndexFile)) {
|
||||
try {
|
||||
const age = Date.now() - statSync(unifiedIndexFile).mtimeMs;
|
||||
if (age < UNIFIED_INDEX_MAX_AGE_MS) {
|
||||
return true;
|
||||
}
|
||||
console.log(
|
||||
`[prebuild] skills-index.json is ${(age / 3600000).toFixed(1)}h old; ` +
|
||||
`refreshing from ${UNIFIED_INDEX_URL}`,
|
||||
);
|
||||
} catch {
|
||||
// fall through to re-fetch
|
||||
}
|
||||
}
|
||||
|
||||
try {
|
||||
const resp = await fetch(UNIFIED_INDEX_URL, {
|
||||
headers: { accept: "application/json" },
|
||||
});
|
||||
if (!resp.ok) {
|
||||
console.warn(
|
||||
`[prebuild] skills-index.json fetch returned HTTP ${resp.status}; ` +
|
||||
`using local copy if any`,
|
||||
);
|
||||
return existsSync(unifiedIndexFile);
|
||||
}
|
||||
const text = await resp.text();
|
||||
// Sanity check: must be valid JSON with a skills array
|
||||
try {
|
||||
const parsed = JSON.parse(text);
|
||||
if (!parsed || !Array.isArray(parsed.skills)) {
|
||||
console.warn(
|
||||
"[prebuild] skills-index.json from live site has no skills array; ignoring",
|
||||
);
|
||||
return existsSync(unifiedIndexFile);
|
||||
}
|
||||
} catch (e) {
|
||||
console.warn(`[prebuild] skills-index.json from live site is not valid JSON: ${e}`);
|
||||
return existsSync(unifiedIndexFile);
|
||||
}
|
||||
mkdirSync(dirname(unifiedIndexFile), { recursive: true });
|
||||
writeFileSync(unifiedIndexFile, text);
|
||||
console.log(
|
||||
`[prebuild] downloaded skills-index.json from ${UNIFIED_INDEX_URL} ` +
|
||||
`(${(text.length / 1024).toFixed(0)} KB)`,
|
||||
);
|
||||
return true;
|
||||
} catch (e) {
|
||||
console.warn(`[prebuild] skills-index.json fetch failed: ${e}`);
|
||||
return existsSync(unifiedIndexFile);
|
||||
}
|
||||
}
|
||||
|
||||
// 0) Pull unified index if we don't have a fresh one.
|
||||
await ensureUnifiedIndex();
|
||||
|
||||
// 1) skills.json — required for the Skills Hub page.
|
||||
if (!existsSync(extractScript)) {
|
||||
writeEmptyFallback("extract script missing");
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue