hermes-agent/.github/workflows/deploy-site.yml
Teknium 5e9d7a7661
fix(skills-hub): stop shipping a degenerate index when GitHub taps collapse (#42347)
The Skills Hub lost every api.github.com-backed source — the OpenAI,
Anthropic, HuggingFace, NVIDIA, gstack, Claude Marketplace and Well-Known
tabs all vanished — while ClawHub/skills.sh/LobeHub/browse.sh survived. A
GitHub API rate limit during the docs-deploy crawl zeroed all three
api.github.com sources (github / claude-marketplace / well-known) at once.

Two compounding bugs let the broken index reach the live site:

1. build_skills_index.py wrote the output file BEFORE the health check, so
   even when the github floor (30) tripped and the script exited 2, the
   degenerate file was already on disk. deploy-site.yml then swallowed the
   exit code with `|| echo non-fatal` and extract-skills.py read the partial
   index. Fix: run the health check first, write the file only when healthy,
   exit without writing on failure. Removed the non-fatal swallow in
   deploy-site.yml so a collapse fails the deploy and the last good site
   stays live (Pages serves the previous build).

2. The build-time GitHub listing path returned [] on a 403 rate-limit without
   retrying or flagging it, so a rate-limited crawl looked identical to an
   empty source. Fix: a shared _github_get() helper on GitHubSource with
   retry/backoff (honors Retry-After / X-RateLimit-Reset on 403/429, backs
   off on 5xx + transport errors) and flags is_rate_limited. Routed
   _list_skills_in_repo and _fetch_file_content through it; gave
   ClaudeMarketplaceSource a persistent GitHubSource + is_rate_limited so the
   builder can name the rate limit as the cause instead of '0 results'.

Added tests/scripts/test_build_skills_index_health.py pinning both contracts:
a degenerate crawl exits non-zero and writes no file; a healthy crawl writes
the index with github/claude-marketplace/well-known all present.
2026-06-08 15:21:28 -07:00

115 lines
4.1 KiB
YAML

name: Deploy Site
on:
release:
types: [published]
push:
branches: [main]
paths:
- 'website/**'
- 'skills/**'
- 'optional-skills/**'
- '.github/workflows/deploy-site.yml'
workflow_dispatch:
permissions:
pages: write
id-token: write
concurrency:
group: pages
cancel-in-progress: false
jobs:
deploy-vercel:
# Triggered automatically on release publish (production cuts) and
# manually via `gh workflow run deploy-site.yml` when an out-of-band
# main commit needs to ship live before the next release tag — e.g.
# a skills-index PR that doesn't touch website/** paths and so
# doesn't auto-deploy via the deploy-docs path.
if: github.event_name == 'release' || github.event_name == 'workflow_dispatch'
runs-on: ubuntu-latest
steps:
- name: Trigger Vercel Deploy
run: curl -X POST "${{ secrets.VERCEL_DEPLOY_HOOK }}"
deploy-docs:
if: github.repository == 'NousResearch/hermes-agent'
runs-on: ubuntu-latest
environment:
name: github-pages
url: ${{ steps.deploy.outputs.page_url }}
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
with:
node-version: 20
cache: npm
cache-dependency-path: website/package-lock.json
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: '3.11'
- name: Install PyYAML for skill extraction
run: pip install pyyaml==6.0.2 httpx==0.28.1
- name: Build skills index (unified multi-source catalog)
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
# Rebuild the unified catalog. The file is gitignored, so a fresh
# checkout starts without it and we want the freshest crawl in
# every deploy.
#
# This MUST be fatal. build_skills_index.py runs a health check and
# exits non-zero WITHOUT writing the output file when a source
# collapses (e.g. a GitHub API rate limit zeroes the github /
# claude-marketplace / well-known taps all at once). Letting the
# deploy continue would either (a) ship a degenerate index missing
# whole hubs — the June 2026 regression where OpenAI/Anthropic/
# HuggingFace/NVIDIA tabs vanished — or (b) fall through to a
# local-only catalog. Failing here keeps the last good deployment
# live (GitHub Pages serves the previous build) instead of
# publishing a broken catalog. Re-run the workflow once the
# transient rate limit clears.
python3 scripts/build_skills_index.py
- name: Extract skill metadata for dashboard
run: python3 website/scripts/extract-skills.py
- name: Regenerate per-skill docs pages + catalogs
run: python3 website/scripts/generate-skill-docs.py
- name: Install dependencies
run: npm ci
working-directory: website
- name: Build Docusaurus
run: npm run build
working-directory: website
- name: Stage deployment
run: |
mkdir -p _site/docs
cp -r website/build/* _site/docs/
# llms.txt / llms-full.txt are also published at the site root
# (https://hermes-agent.nousresearch.com/llms.txt) because some
# agents and IDE plugins probe the classic root-level path rather
# than /docs/llms.txt. Same file, two URLs, one source of truth.
if [ -f website/build/llms.txt ]; then
cp website/build/llms.txt _site/llms.txt
fi
if [ -f website/build/llms-full.txt ]; then
cp website/build/llms-full.txt _site/llms-full.txt
fi
- name: Upload artifact
uses: actions/upload-pages-artifact@56afc609e74202658d3ffba0e8f6dda462b719fa # v3
with:
path: _site
- name: Deploy to GitHub Pages
id: deploy
uses: actions/deploy-pages@d6db90164ac5ed86f2b6aed7e0febac5b3c0c03e # v4