mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-07-21 16:18:55 +00:00
* feat(attribution): conflict-free contributor mappings via contributors/emails/ directory
The AUTHOR_MAP dict in scripts/release.py was a merge-conflict magnet:
every concurrent salvage PR appended entries to the same lines of the
same file, so parallel PRs re-conflicted on every merge to main.
New system: one file per email under contributors/emails/ — filename is
the commit-author email, first non-comment line is the GitHub login.
File additions never conflict, so any number of PRs can add mappings
concurrently.
- scripts/release.py: AUTHOR_MAP is now LEGACY_AUTHOR_MAP (frozen)
merged with the directory at import time (directory wins). All
existing consumers (resolve_author, contributor_audit.py) unchanged.
- scripts/add_contributor.py: idempotent CLI to add a mapping; refuses
conflicting reassignments (incl. against the legacy map), validates
email/login shapes.
- contributor-check.yml: attribution gate now accepts a mapping file OR
a legacy entry; failure message prints the exact add_contributor
command. Also auto-resolves bare <login>@users.noreply.github.com
emails is intentionally NOT added (kept id+login form only, matching
previous behavior).
- contributor_audit.py: guidance now points at add_contributor.py.
- tests/scripts/test_contributor_map.py: 12 tests covering loader,
merge precedence, CLI idempotency/conflict/validation, subprocess E2E.
* feat(ci): one-shot per-file flake retry in the parallel test runner
A failing test FILE is re-run once in a fresh subprocess. Pass-on-retry
counts as green but is loudly reported in a '⚠ FLAKY' summary section
(with both attempts' output preserved) so the flake gets fixed instead
of eating a full-run rerun. Deterministic failures fail both attempts —
regressions cannot be laundered green.
- --file-retries N / HERMES_TEST_FILE_RETRIES (default 1, 0 disables)
- E2E verified: simulated first-run-fail flake goes green with banner;
deterministic failure still exits 1; retries=0 restores old behavior.
This converts the dominant CI failure mode (one timing-sensitive test
flaking a 4600-test shard, requiring a manual 10-minute rerun and an
agent triage loop) into a self-healing retry that costs one file's
runtime.
* test(approval): loosen wall-clock perf bounds 0.15s -> 2.0s
These guard against catastrophic regex backtracking (seconds-to-minutes
class), but 0.15s is within scheduler-stall noise on loaded shared CI
runners — test_max_accepted_separator_free_input_is_fast failed a CI
shard this week on runner load alone. 2.0s still catches the regression
class with zero flake surface.
* fix(ci): job timeouts everywhere + retries on all network installs
Reliability pass over every workflow:
- timeout-minutes on all 21 jobs that lacked one (a hung job previously
burned the 6-hour default runner budget)
- ./.github/actions/retry wrapped around every network-fetching install
that lacked it: pip installs (deploy-site, skills-index), npm ci
(deploy-site website, upload_to_pypi web + ui-tui), uv sync (docker
test deps). Deterministic build steps (npm run build) deliberately
NOT retried — split into separate steps so a real build failure fails
fast instead of retrying 3x.
* docs(agents): document the file-retry flake policy
* fix(ci): curl retries on deploy hook + skills-index probe
* fix(ci): kill the remaining transient-failure classes in workflows + Dockerfile
From the workflow reliability audit:
- tests.yml: duration-cache restore had NO restore-keys while saves use
run_id-suffixed keys — the cache never matched once, so LPT slicing
always ran blind and unbalanced slices pushed heavy files toward the
per-file timeout. One-line restore-keys fixes slice balancing.
- Label gates (lint ci-reviewed, supply-chain mcp-catalog-reviewed):
'gh pr view || true' turned an API blip into 'label absent' → false
BLOCKING failure. Now 3x retry, and API failure is reported as an API
failure instead of a missing label.
- detect-changes action: compare API retried before failing open (was
silently running all lanes on any blip).
- uv-lockfile-check: 'uv lock --check' resolves against PyPI — retried
so registry blips don't read as 'lockfile stale'.
- docker.yml merge job: imagetools create retried (Docker Hub eventual
consistency on just-pushed digests).
- Dockerfile: apt-get Acquire::Retries=3; s6-overlay ADDs converted to
curl --retry 3 (ADD cannot retry; checksums still enforced); npm
--fetch-retries=5; playwright chromium fetch retried 3x.
- Advisory artifact uploads (per-slice durations, ci-timings report)
get continue-on-error so an artifact-service blip can't fail a green
test slice.
* fix(tests): kill the two root-cause flakes — leaking pre-warm timer + env-dependent provider list
- test_tui_gateway_server.py: session.create / non-eager session.resume
arm a 50ms threading.Timer (_schedule_agent_build) that outlives its
test and fires into the NEXT test's _make_agent mock, racily
corrupting captured state (the recurring session_resume shard
failures). Replaced the per-test whack-a-mole stub with a module-wide
autouse fixture; the 3 worker-lifecycle tests that genuinely need the
deferred build opt back in via @pytest.mark.real_agent_prewarm (new
marker in pyproject).
- test_api_key_providers.py: PROVIDER_ENV_VARS is now derived from the
live PROVIDER_REGISTRY instead of a hand-list that had drifted
(missing HF_TOKEN / DEEPINFRA_API_KEY) — resolve_provider('auto')
tests failed on any machine with HF_TOKEN exported. E2E-verified with
HF_TOKEN/DEEPINFRA_API_KEY set: 42/42 pass.
* test: de-flake 30 timing-sensitive test files for loaded CI runners
Root-cause fixes from the flake audit (session-DB mining + repo sweep):
Event-based sync instead of sleep-sync:
- title_generator: mock sets threading.Event, wait(10) replaces
sleep(0.3) hoping the daemon thread got scheduled
- docker zombie_reaping / profile_gateway: poll-for-state helpers
replace fixed 1-3s sleeps (s6 transitions + SIGCHLD reaping are async)
- process_registry tree test: select()-bounded readline replaces an
unbounded blocking read (parent wedge now fails THIS test with a clear
message instead of an opaque rc=124 file kill); SIGTERM grace 1s->2s
(the 1s partition window mid-interpreter-startup is how a child PID
escaped the live-system guard in CI)
Timeout raises (loaded 8-way-sliced runners see ~5s scheduling floors;
all of these complete in ms-to-1s when healthy so the raises cost
nothing on green runs):
- subprocess/thread waits <= 2s raised to 10-15s across mcp_tool,
mcp_circuit_breaker, mcp_reconnect_retry_reset, mcp_parked_self_probe,
mcp_cancelled_error_propagation, registry, clarify_gateway, interrupt,
voice_cli_integration, docker_environment, session_store_lock_io,
planned_stop_watcher, cli_interrupt_subagent, thread_scoped_output
(joins now also assert not is_alive() so stragglers fail loudly)
- wall-clock discrimination ceilings loosened where the guarded hang is
10x larger: local_background_child_hang 4s->10s, interrupt_cleanup
setup 5s->20s + pgid-exit 30s->60s, mcp_stability grandchild spinup
5s->15s, protocol/gil-starvation fast-handler 0.5s->2s,
iso_certify_seam 1.5s->5s, wait_for_mcp_discovery 0.1s->1s
- narrow assertion windows widened: honcho first-turn wait 0.4..0.65 ->
0.25..2.0 (property is bounded-not-hung, not an exact wall-clock);
compression fork-lock TTL 1s->3s (12 refresh chances per lease);
compression-lock expiry margins symmetric (ttl 0.05->0.5, sleep 1.0)
- telegram hung-DNS bound 1.0->1.4 (fake hang is 1.5s — must stay under)
* fix(tests): repair indentation from de-flake batch edit
* fix(tests): harden env isolation and replace remaining sleep-sync races
The full 42k-test run and complete npm check surfaced three more classes:
- Environment isolation: local ~/.honcho defaultHost and SSH_* variables
leaked into Python/TUI tests. Pin the default Honcho host in the
hermetic fixture, isolate the one fallback test from ~/.honcho, and
blank SSH_* around terminalSetup tests. This flipped 20 false failures
back to deterministic behavior on developer machines.
- Background-thread sleep-sync: Honcho async writer tests patched
time.sleep globally, then busy-polled with that same mocked sleep. Under
full-suite load the poller could starve the writer. Each test now waits
on an Event emitted by the exact flush/retry transition; 30/30 passed
under 15-way contention.
- Desktop streaming: the test slept 80ms and assumed a 500ms timer could
not fire before its assertion. A loaded runner descheduled the test for
>500ms and both chunks arrived. Producer controls now gate second-chunk
and completion transitions explicitly.
Also make file-retry observability complete: a self-healed flaky file now
prints BOTH attempts' full output in the FLAKY summary. Two behavioral
runner tests prove pass-on-retry is green+loud+traceback-preserving, while
a deterministic failure remains red.
* refactor(ci): use gh bot pat, better retries
refactor(ci): use retry action for PR label fetch
the retry action now captures stdout as a step output, so it can serve
double duty: retry + output capture for commands like 'gh pr view' whose
result must be consumed by later steps.
Retry action gains:
- 'stdout' output (heredoc-delimited to preserve newlines)
- tee to temp file so stdout still streams to the job log
- step id 'retry' for output reference
Both lint.yml and supply-chain-audit.yml now use the retry action
directly with 'command: gh pr view ...' and read
steps.<id>.outputs.stdout.
ci: use AUTOFIX_BOT_PAT for all gh CLI / GitHub API auth
Replace secrets.GITHUB_TOKEN and github.token with
secrets.AUTOFIX_BOT_PAT across all workflows and composite actions
that use the gh CLI or GitHub API. The PAT has consistent permissions
across fork PRs (where GITHUB_TOKEN is read-only), avoids API rate
limit sharing with the default token, and is already used by
js-autofix.yml for the same reasons.
19 sites swapped across 9 files:
- lint.yml (3): label fetch, comment post/edit, comment update
- supply-chain-audit.yml (5): scan, critical comment, unbounded dep
comment, label fetch, mcp-catalog comment
- lockfile-diff.yml (1): PR comment post/update
- skills-index-freshness.yml (1): issue creation on degraded probe
- skills-index.yml (2): index build, trigger deploy workflow
- upload_to_pypi.yml (2): release view poll, release upload
- ci.yml (1): timings report
- deploy-site.yml (2): skills index crawl
- detect-changes/action.yml (1): compare API call
---------
Co-authored-by: ethernet <arilotter@gmail.com>
275 lines
11 KiB
YAML
275 lines
11 KiB
YAML
name: Lint (ruff + ty)
|
|
|
|
# Two things here:
|
|
# 1. Advisory diff — ruff + ty diagnostics as a diff vs the target branch.
|
|
# Posts a Markdown summary and a PR comment. Exit zero always.
|
|
# 2. Blocking ``ruff check .`` — enforces the explicit rules in
|
|
# ``[tool.ruff.lint.select]`` (currently PLW1514). Failure blocks merge.
|
|
# Separate job so the advisory diff still runs and posts even when
|
|
# enforcement fails.
|
|
|
|
on:
|
|
workflow_call:
|
|
inputs:
|
|
event_name:
|
|
description: The event name from the calling orchestrator (pull_request or push).
|
|
type: string
|
|
required: true
|
|
ci_review:
|
|
description: Whether CI-sensitive files (eslint config, workflows, actions) changed and require a review label.
|
|
type: boolean
|
|
default: false
|
|
|
|
permissions:
|
|
contents: read
|
|
pull-requests: write # needed to post/update PR comments
|
|
|
|
concurrency:
|
|
group: lint-${{ github.ref }}
|
|
cancel-in-progress: true
|
|
|
|
jobs:
|
|
lint-diff:
|
|
name: ruff + ty diff
|
|
if: inputs.event_name == 'pull_request'
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 10
|
|
steps:
|
|
- name: Checkout code
|
|
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
|
with:
|
|
fetch-depth: 0 # need full history for merge-base + worktree
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # 8.2.0
|
|
|
|
- name: Install ruff + ty
|
|
uses: ./.github/actions/retry
|
|
with:
|
|
command: uv tool install ruff && uv tool install ty
|
|
|
|
- name: Determine base ref
|
|
id: base
|
|
run: |
|
|
# For PRs, diff against the merge base with the target branch.
|
|
# For pushes to main, diff against the previous commit on main.
|
|
if [ "${{ inputs.event_name }}" = "pull_request" ]; then
|
|
BASE_SHA=$(git merge-base "origin/${{ github.base_ref }}" HEAD)
|
|
BASE_REF="origin/${{ github.base_ref }}"
|
|
else
|
|
BASE_SHA=$(git rev-parse HEAD~1 2>/dev/null || git rev-parse HEAD)
|
|
BASE_REF="HEAD~1"
|
|
fi
|
|
echo "sha=${BASE_SHA}" >> "$GITHUB_OUTPUT"
|
|
echo "ref=${BASE_REF}" >> "$GITHUB_OUTPUT"
|
|
echo "Base SHA: ${BASE_SHA}"
|
|
echo "Base ref: ${BASE_REF}"
|
|
|
|
- name: Run ruff + ty on HEAD
|
|
run: |
|
|
mkdir -p .lint-reports/head
|
|
ruff check --output-format json --exit-zero \
|
|
> .lint-reports/head/ruff.json || true
|
|
ty check --output-format gitlab --exit-zero \
|
|
> .lint-reports/head/ty.json || true
|
|
echo "HEAD ruff: $(wc -c < .lint-reports/head/ruff.json) bytes"
|
|
echo "HEAD ty: $(wc -c < .lint-reports/head/ty.json) bytes"
|
|
|
|
- name: Run ruff + ty on base (via git worktree)
|
|
run: |
|
|
mkdir -p .lint-reports/base
|
|
# Use a worktree so we don't clobber the main checkout. If the basex
|
|
# SHA is identical to HEAD (e.g. first commit), skip and leave the
|
|
# base reports empty — the diff script handles missing files.
|
|
HEAD_SHA=$(git rev-parse HEAD)
|
|
BASE_SHA="${{ steps.base.outputs.sha }}"
|
|
if [ "$BASE_SHA" = "$HEAD_SHA" ]; then
|
|
echo "Base SHA == HEAD SHA, skipping base scan."
|
|
echo '[]' > .lint-reports/base/ruff.json
|
|
echo '[]' > .lint-reports/base/ty.json
|
|
else
|
|
git worktree add --detach /tmp/lint-base "$BASE_SHA"
|
|
(
|
|
cd /tmp/lint-base
|
|
ruff check --output-format json --exit-zero \
|
|
> "$GITHUB_WORKSPACE/.lint-reports/base/ruff.json" || true
|
|
ty check --output-format gitlab --exit-zero \
|
|
> "$GITHUB_WORKSPACE/.lint-reports/base/ty.json" || true
|
|
)
|
|
git worktree remove --force /tmp/lint-base
|
|
fi
|
|
echo "base ruff: $(wc -c < .lint-reports/base/ruff.json) bytes"
|
|
echo "base ty: $(wc -c < .lint-reports/base/ty.json) bytes"
|
|
|
|
- name: Generate diff summary
|
|
env:
|
|
HEAD_REF: ${{ inputs.event_name == 'pull_request' && github.head_ref || github.ref_name }}
|
|
run: |
|
|
python scripts/lint_diff.py \
|
|
--base-ruff .lint-reports/base/ruff.json \
|
|
--head-ruff .lint-reports/head/ruff.json \
|
|
--base-ty .lint-reports/base/ty.json \
|
|
--head-ty .lint-reports/head/ty.json \
|
|
--base-ref "${{ steps.base.outputs.ref }}" \
|
|
--head-ref "$HEAD_REF" \
|
|
--output .lint-reports/summary.md
|
|
cat .lint-reports/summary.md >> "$GITHUB_STEP_SUMMARY"
|
|
|
|
ruff-blocking:
|
|
# Enforce the rules in pyproject.toml [tool.ruff.lint.select]. Currently
|
|
# PLW1514 (unspecified-encoding) — catches bare ``open()`` /
|
|
# ``read_text()`` / ``write_text()`` calls that default to locale
|
|
# encoding on Windows. Failure here blocks merge; the advisory
|
|
# ``lint-diff`` job above runs independently so reviewers still get
|
|
# the diff comment even when enforcement fails.
|
|
name: ruff enforcement (blocking)
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 5
|
|
steps:
|
|
- name: Checkout code
|
|
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # 8.2.0
|
|
|
|
- name: Install ruff
|
|
uses: ./.github/actions/retry
|
|
with:
|
|
command: uv tool install ruff
|
|
|
|
- name: ruff check .
|
|
# No --exit-zero, no || true. Exit code propagates to the job,
|
|
# which propagates to the required-check gate.
|
|
run: |
|
|
ruff check .
|
|
|
|
windows-footguns:
|
|
# Static guardrails on Windows-unsafe Python primitives — os.kill(pid, 0),
|
|
# os.killpg, os.setsid, signal.SIGKILL without getattr fallback,
|
|
# shebang scripts via subprocess, bare open() without encoding=, etc.
|
|
# See scripts/check-windows-footguns.py for the full rule list.
|
|
name: Windows footguns (blocking)
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 5
|
|
steps:
|
|
- name: Checkout code
|
|
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v5
|
|
with:
|
|
python-version: "3.11"
|
|
|
|
- name: Run footgun checker
|
|
run: python scripts/check-windows-footguns.py --all
|
|
|
|
ci-review:
|
|
# Require explicit maintainer review when CI-sensitive files change:
|
|
# eslint config, workflow YAMLs, or composite actions. These files
|
|
# influence what code the js-autofix job executes and pushes to
|
|
# main, so a malicious PR could inject arbitrary code via a custom eslint
|
|
# rule's `fix` function. The label gate ensures a human reviews before
|
|
# merge. Mirrors the mcp-catalog-reviewed pattern in supply-chain-audit.yml.
|
|
name: CI-sensitive file review
|
|
if: inputs.event_name == 'pull_request' && inputs.ci_review
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 2
|
|
steps:
|
|
- name: Checkout code
|
|
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
|
- name: Fetch PR labels
|
|
id: pr-labels
|
|
uses: ./.github/actions/retry
|
|
env:
|
|
GH_TOKEN: ${{ secrets.AUTOFIX_BOT_PAT }}
|
|
with:
|
|
command: gh pr view "${{ github.event.pull_request.number }}" --json labels --jq '.labels[].name'
|
|
- name: Require ci-reviewed label
|
|
id: label-check
|
|
run: |
|
|
set -euo pipefail
|
|
if echo "${{ steps.pr-labels.outputs.stdout }}" | grep -Fxq 'ci-reviewed'; then
|
|
echo "reviewed=true" >> "$GITHUB_OUTPUT"
|
|
echo "ci-reviewed label present."
|
|
exit 0
|
|
fi
|
|
echo "reviewed=false" >> "$GITHUB_OUTPUT"
|
|
|
|
# On failure: find the bot's previous comment and edit it, or create
|
|
# a new one if none exists. Using an HTML comment marker so we can
|
|
# locate it reliably across runs without parsing the body text.
|
|
# Skipped on fork PRs — GITHUB_TOKEN is read-only there, so the API
|
|
# call would fail. The label gate still holds via the step below.
|
|
- name: Post or update review warning
|
|
if: steps.label-check.outputs.reviewed != 'true' && github.event.pull_request.head.repo.fork != true
|
|
env:
|
|
GH_TOKEN: ${{ secrets.AUTOFIX_BOT_PAT }}
|
|
run: |
|
|
set -euo pipefail
|
|
PR="${{ github.event.pull_request.number }}"
|
|
MARKER="<!-- ci-review-bot -->"
|
|
BODY="${MARKER}
|
|
## ⚠️ CI-sensitive file review required
|
|
|
|
This PR changes CI-sensitive files (eslint config, workflow YAMLs,
|
|
or composite actions). These files influence what code the
|
|
js-autofix job executes and pushes to main.
|
|
|
|
A maintainer should verify:
|
|
- no new eslint rules with custom \`fix\` functions that write outside linted paths,
|
|
- no workflow changes that widen permissions or remove guards,
|
|
- no composite action changes that alter what gets executed.
|
|
|
|
After review, add the \`ci-reviewed\` label and re-run this check."
|
|
|
|
# Find an existing comment with our marker.
|
|
COMMENT_ID=$(gh api \
|
|
"repos/${{ github.repository }}/issues/${PR}/comments" \
|
|
--paginate --jq ".[] | select(.body | contains(\"${MARKER}\")) | .id" \
|
|
| head -1 || true)
|
|
|
|
if [ -n "$COMMENT_ID" ]; then
|
|
gh api --method PATCH \
|
|
"repos/${{ github.repository }}/issues/comments/${COMMENT_ID}" \
|
|
-f body="$BODY"
|
|
else
|
|
gh pr comment "$PR" --body "$BODY"
|
|
fi
|
|
|
|
# Fail the job when the label is missing — always runs (including
|
|
# fork PRs) so the security gate holds even when the comment step
|
|
# was skipped above.
|
|
- name: Fail on missing label
|
|
if: steps.label-check.outputs.reviewed != 'true'
|
|
run: |
|
|
echo "::error::CI-sensitive changes require the ci-reviewed label."
|
|
exit 1
|
|
|
|
# On success: if a previous warning comment exists, edit it to show
|
|
# the review passed so the PR doesn't have a stale ⚠️ sitting around.
|
|
# Skipped on fork PRs — no comment was ever posted to update.
|
|
- name: Update previous warning to passed
|
|
if: steps.label-check.outputs.reviewed == 'true' && github.event.pull_request.head.repo.fork != true
|
|
env:
|
|
GH_TOKEN: ${{ secrets.AUTOFIX_BOT_PAT }}
|
|
run: |
|
|
set -euo pipefail
|
|
PR="${{ github.event.pull_request.number }}"
|
|
MARKER="<!-- ci-review-bot -->"
|
|
|
|
# Find an existing comment with our marker.
|
|
COMMENT_ID=$(gh api \
|
|
"repos/${{ github.repository }}/issues/${PR}/comments" \
|
|
--paginate --jq ".[] | select(.body | contains(\"${MARKER}\")) | .id" \
|
|
| head -1 || true)
|
|
|
|
if [ -n "$COMMENT_ID" ]; then
|
|
BODY="${MARKER}
|
|
## ✅ CI-sensitive file review passed
|
|
|
|
The \`ci-reviewed\` label is present on this PR."
|
|
|
|
gh api --method PATCH \
|
|
"repos/${{ github.repository }}/issues/comments/${COMMENT_ID}" \
|
|
-f body="$BODY"
|
|
fi
|