mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-07-26 17:38:36 +00:00
ci: live-updating PR review comment with structured job statuses
Replace the static comment-pending + comment-results two-job pattern
with a live-updating comment system that polls the GitHub Actions API
every 15s, re-assembles the review comment from whatever results are
available, and upserts it via the <!-- hermes-ci-review-bot --> marker.
The comment updates in real time as each job finishes — no waiting for
the full pipeline.
Every CI job that wants to appear in the review comment emits a
review_status output — a JSON array of objects, each with a source
and a results array:
[
{
"source": "review-label-gate",
"results": [
{"kind": "action_required", "title": "...", "summary": "...",
"how_to_fix": "..."},
{"kind": "info", "title": "...", "summary": "..."}
]
},
{
"source": "ci timing",
"results": [
{"kind": "warning", "title": "CI timings", "summary": "...",
"detail": "...", "link": "..."}
]
}
]
One job can emit multiple results of different kinds. The source field
is used to exclude the corresponding job from the synthesized error
list (case-insensitive, hyphen-normalized matching against GitHub
Actions job display names).
| job | source | kind (on failure) | section |
|----------------------------|--------------------------|---------------------------|----------------------|
| review-labels | review label gate | action_required / info | Action required |
| lockfile-diff | lockfile-diff | action_required | Action required |
| ci-timings | ci timing | warning / info | Warnings |
| supply-chain scan | supply chain | error / (none) | Job failures |
| supply-chain dep-bounds | supply chain | action_required / (none) | Action required |
| osv-scanner | osv scan | warning / (none) | Warnings |
| uv-lockfile-check | uv.lock check | action_required / (none) | Action required |
| history-check | unrelated histories | action_required | Action required |
| contributor-check | contributor attribution | action_required | Action required |
Jobs that find nothing emit [] (empty array) — no noise info items.
A single comment-live job polls the GitHub Actions API every 15s,
classifies jobs into (completed, pending), assembles the comment, and
upserts it. Merges review_status outputs from all needs jobs via
toJSON(needs.*.outputs.review_status), and downloads the ci-timings
artifact when it becomes available. Shows commit SHA + message below
the header.
The assembler has ZERO job-specific knowledge. It just:
1. collect_from_statuses() — flattens all nested status objects into ReviewItems
2. collect_failed_jobs() — synthesizes errors for failed jobs with no declared status
3. _attach_job_urls() — fills in per-job log links for ALL items
4. render_comment() — groups by severity, renders with group headers
Each item shows links inline next to the title: View report (job-emitted
URL) and View job (auto-attached logs link). Each info item is its own
collapsible <details> block.
# ૮ >ﻌ< ა ci review
running on abc1234 — commit message first line
## ❌ Job failures
### {title} · [View job](url)
{summary}
## ⚠️ Action required
### {title} · [View job](url)
{summary}
**How to fix:**
{how_to_fix}
## ⚠️ Warnings
### {title} · [View report](url) · [View job](url)
{summary}
{detail}
<details><summary>{title}</summary>
{content}
</details>
Still running 3 jobs: ci-timings, docker
- test_assemble_review_comment.py (48 tests): collect_from_statuses,
collect_failed_jobs with exclude_sources, _attach_job_urls,
render_comment (group headers, inline links, commit info, per-item
details, pending footer), assemble integration
- test_live_comment.py (16 tests): classify_jobs pure function
- test_timings_report.py (10 tests): generate_review_status nested format
- test_lockfile_diff.py (6 tests)
- test_classify_changes.py (32 tests, pre-existing)
This commit is contained in:
parent
d7b36070ef
commit
b9f82ed39f
19 changed files with 2696 additions and 323 deletions
116
.github/workflows/ci.yml
vendored
116
.github/workflows/ci.yml
vendored
|
|
@ -17,7 +17,7 @@ on:
|
|||
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: write # needed by lint (PR comment) + supply-chain (PR comment)
|
||||
pull-requests: write # needed by lint (PR comment) + supply-chain review_status
|
||||
actions: read # needed by osv-scanner (SARIF upload)
|
||||
security-events: write # needed by osv-scanner (SARIF upload)
|
||||
packages: write # needed by docker build
|
||||
|
|
@ -73,11 +73,10 @@ jobs:
|
|||
lint:
|
||||
name: Python lints
|
||||
needs: detect
|
||||
if: needs.detect.outputs.python == 'true' || needs.detect.outputs.ci_review == 'true'
|
||||
if: needs.detect.outputs.python == 'true'
|
||||
uses: ./.github/workflows/lint.yml
|
||||
with:
|
||||
event_name: ${{ needs.detect.outputs.event_name }}
|
||||
ci_review: ${{ needs.detect.outputs.ci_review == 'true' }}
|
||||
secrets: inherit
|
||||
|
||||
js-tests:
|
||||
|
|
@ -144,12 +143,20 @@ jobs:
|
|||
supply-chain:
|
||||
name: Supply-chain scan
|
||||
needs: detect
|
||||
if: needs.detect.outputs.event_name == 'pull_request' && (needs.detect.outputs.scan == 'true' || needs.detect.outputs.deps == 'true' || needs.detect.outputs.mcp_catalog == 'true')
|
||||
if: needs.detect.outputs.event_name == 'pull_request' && (needs.detect.outputs.scan == 'true' || needs.detect.outputs.deps == 'true')
|
||||
uses: ./.github/workflows/supply-chain-audit.yml
|
||||
with:
|
||||
event_name: ${{ needs.detect.outputs.event_name }}
|
||||
scan: ${{ needs.detect.outputs.scan == 'true' }}
|
||||
deps: ${{ needs.detect.outputs.deps == 'true' }}
|
||||
|
||||
review-labels:
|
||||
name: Review label gate
|
||||
needs: detect
|
||||
if: needs.detect.outputs.event_name == 'pull_request' && (needs.detect.outputs.ci_review == 'true' || needs.detect.outputs.mcp_catalog == 'true')
|
||||
uses: ./.github/workflows/review-labels.yml
|
||||
with:
|
||||
ci_review: ${{ needs.detect.outputs.ci_review == 'true' }}
|
||||
mcp_catalog: ${{ needs.detect.outputs.mcp_catalog == 'true' }}
|
||||
secrets: inherit
|
||||
|
||||
|
|
@ -158,12 +165,90 @@ jobs:
|
|||
uses: ./.github/workflows/osv-scanner.yml
|
||||
secrets: inherit
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
# Live-updating PR review comment.
|
||||
#
|
||||
# A single ``comment-live`` job polls the GitHub Actions API every 15s
|
||||
# for job statuses in this run, re-assembles the review comment from
|
||||
# whatever results are available, and upserts it via the
|
||||
# ``<!-- hermes-ci-review-bot -->`` marker.
|
||||
#
|
||||
# The poller exits when all non-infra jobs are completed (or on
|
||||
# timeout). ci-timings' review_status is picked up automatically when
|
||||
# its artifact becomes available — the poller downloads and merges it.
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
comment-live:
|
||||
name: CI review comment (live)
|
||||
needs: [detect, review-labels, lockfile-diff, supply-chain, osv-scanner, uv-lockfile, history-check, contributor-check]
|
||||
if: always() && github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork != true
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 40
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
|
||||
- name: Run live comment poller
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
GITHUB_RUN_ID: ${{ github.run_id }}
|
||||
PR_NUMBER: ${{ github.event.pull_request.number }}
|
||||
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
|
||||
# Commit info for the review comment header.
|
||||
COMMIT_SHA: ${{ github.event.pull_request.head.sha }}
|
||||
COMMIT_MESSAGE: ${{ github.event.pull_request.head.commit.message }}
|
||||
COMMIT_URL: ${{ github.server_url }}/${{ github.repository }}/pull/${{ github.event.pull_request.number }}/commits/${{ github.event.pull_request.head.sha }}
|
||||
# Structured review statuses from workflow_call jobs.
|
||||
# Each job outputs a JSON array of {source, results: [...]} objects
|
||||
# that the assembler renders directly — no hardcoded job-name
|
||||
# matching. We merge all available outputs into one array.
|
||||
REVIEW_STATUSES: ${{ toJSON(needs.*.outputs.review_status) }}
|
||||
run: |
|
||||
set -uo pipefail
|
||||
|
||||
# REVIEW_STATUSES is a JSON array of strings (some may be empty
|
||||
# when a job was skipped). Parse each string and merge into one
|
||||
# flat array for the assembler.
|
||||
python3 - <<'PYEOF'
|
||||
import json, os, sys
|
||||
|
||||
raw = os.environ.get("REVIEW_STATUSES", "")
|
||||
merged = []
|
||||
if raw:
|
||||
try:
|
||||
arr = json.loads(raw)
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
arr = []
|
||||
for item in arr:
|
||||
if not item:
|
||||
continue
|
||||
try:
|
||||
statuses = json.loads(item)
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
continue
|
||||
if isinstance(statuses, list):
|
||||
merged.extend(statuses)
|
||||
|
||||
# Write merged array to a temp file the poller reads.
|
||||
with open("/tmp/review_statuses.json", "w") as f:
|
||||
json.dump(merged, f)
|
||||
print(f"Merged {len(merged)} review status entries")
|
||||
PYEOF
|
||||
|
||||
python3 scripts/ci/live_comment.py \
|
||||
--interval 15 \
|
||||
--timeout 2100 \
|
||||
--review-statuses-file /tmp/review_statuses.json
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
# Gate: runs after everything. ``if: always()`` ensures it reports a
|
||||
# status even when some deps were skipped. Only actual ``failure``
|
||||
# results cause it to fail; ``skipped`` is treated as success.
|
||||
#
|
||||
# Branch protection should require ONLY this check.
|
||||
#
|
||||
# Outputs ``needs-json`` — a compact ``{job_name: result}`` dict — so
|
||||
# the live comment poller can list failed jobs in the PR comment.
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
all-checks-pass:
|
||||
name: All required checks pass
|
||||
|
|
@ -179,20 +264,30 @@ jobs:
|
|||
- lockfile-diff
|
||||
- docker-lint
|
||||
- supply-chain
|
||||
- review-labels
|
||||
- osv-scanner
|
||||
# comment-live is a polling job — it doesn't block the gate.
|
||||
# we don't require docker to pass rn because it's so slow lol
|
||||
# - docker
|
||||
if: always()
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
outputs:
|
||||
needs-json: ${{ steps.evaluate.outputs.needs-json }}
|
||||
steps:
|
||||
- name: Evaluate job results
|
||||
id: evaluate
|
||||
env:
|
||||
NEEDS: ${{ toJSON(needs) }}
|
||||
run: |
|
||||
echo "$NEEDS" | python3 -c "
|
||||
import json, sys
|
||||
needs = json.load(sys.stdin)
|
||||
# Emit compact {job_name: result} for the comment assembler.
|
||||
compact = {name: info['result'] for name, info in needs.items()}
|
||||
print(f'needs-json={json.dumps(compact)}')
|
||||
with open('$GITHUB_OUTPUT', 'a') as f:
|
||||
f.write(f'needs-json={json.dumps(compact)}\n')
|
||||
failed = [name for name, info in needs.items() if info['result'] == 'failure']
|
||||
for name, info in sorted(needs.items()):
|
||||
result = info['result']
|
||||
|
|
@ -209,6 +304,9 @@ jobs:
|
|||
# cache them on main (as a baseline), and on PRs generate an HTML diff
|
||||
# report with a gantt chart + per-step breakdown. The report is uploaded
|
||||
# as an artifact and a markdown summary is written to $GITHUB_STEP_SUMMARY.
|
||||
#
|
||||
# The live comment poller picks up ci-timings' completion automatically —
|
||||
# it reads review-status.json from the artifact when the job finishes.
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
ci-timings:
|
||||
name: CI timing report
|
||||
|
|
@ -242,18 +340,20 @@ jobs:
|
|||
--baseline ci-timings-baseline.json \
|
||||
--output ci-timings-report.html \
|
||||
--json-out ci-timings.json \
|
||||
--summary-out ci-timings-summary.md
|
||||
--summary-out ci-timings-summary.md \
|
||||
--review-status-out review-status.json
|
||||
|
||||
- name: Upload HTML report
|
||||
- name: Upload HTML report + review status
|
||||
# Advisory report — artifact-service blips must not fail the job.
|
||||
continue-on-error: true
|
||||
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
||||
id: ci-timings-artifact
|
||||
with:
|
||||
name: ci-timings-report
|
||||
path: ci-timings-report.html
|
||||
path: |
|
||||
ci-timings-report.html
|
||||
review-status.json
|
||||
retention-days: 14
|
||||
archive: false
|
||||
|
||||
- name: Output summary
|
||||
env:
|
||||
|
|
|
|||
18
.github/workflows/contributor-check.yml
vendored
18
.github/workflows/contributor-check.yml
vendored
|
|
@ -2,6 +2,10 @@ name: Contributor Attribution Check
|
|||
|
||||
on:
|
||||
workflow_call:
|
||||
outputs:
|
||||
review_status:
|
||||
description: "JSON array of review status objects"
|
||||
value: ${{ jobs.check-attribution.outputs.review_status }}
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
|
@ -10,12 +14,15 @@ jobs:
|
|||
check-attribution:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
outputs:
|
||||
review_status: ${{ steps.check-emails.outputs.review_status }}
|
||||
steps:
|
||||
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
fetch-depth: 0 # Full history needed for git log
|
||||
|
||||
- name: Check for unmapped contributor emails
|
||||
id: check-emails
|
||||
run: |
|
||||
# Get the merge base between this PR and main
|
||||
MERGE_BASE=$(git merge-base origin/main HEAD)
|
||||
|
|
@ -25,6 +32,7 @@ jobs:
|
|||
|
||||
if [ -z "$NEW_EMAILS" ]; then
|
||||
echo "No new commits to check."
|
||||
echo "review_status=[]" >> "$GITHUB_OUTPUT"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
|
|
@ -67,6 +75,16 @@ jobs:
|
|||
echo ""
|
||||
echo "To find the GitHub username for an email:"
|
||||
echo " gh api 'search/users?q=EMAIL+in:email' --jq '.items[0].login'"
|
||||
|
||||
# Emit review_status for unmapped emails
|
||||
DETAIL=$(echo -e "$MISSING" | sed '/^$/d; s/^ //')
|
||||
HOW_TO_FIX=$'Add mappings to scripts/release.py AUTHOR_MAP:\n```\n"<email>": "<github-username>",\n```\nTo find the GitHub username for an email:\n```\ngh api \'search/users?q=EMAIL+in:email\' --jq \'.items[0].login\'\n```\n'
|
||||
REVIEW_STATUS=$(jq -nc \
|
||||
--arg detail "$DETAIL" \
|
||||
--arg how_to_fix "$HOW_TO_FIX" \
|
||||
'[{"source":"contributor attribution","results":[{"kind":"action_required","title":"Unmapped contributor email(s)","summary":"New contributor email(s) are not in AUTHOR_MAP.","detail":$detail,"how_to_fix":$how_to_fix}]}]')
|
||||
echo "review_status=$REVIEW_STATUS" >> "$GITHUB_OUTPUT"
|
||||
|
||||
exit 1
|
||||
else
|
||||
echo "✅ All contributor emails are mapped."
|
||||
|
|
|
|||
12
.github/workflows/history-check.yml
vendored
12
.github/workflows/history-check.yml
vendored
|
|
@ -15,6 +15,10 @@ name: History Check
|
|||
|
||||
on:
|
||||
workflow_call:
|
||||
outputs:
|
||||
review_status:
|
||||
description: "JSON array of review_status objects for the synthesizer."
|
||||
value: ${{ jobs.check-common-ancestor.outputs.review_status }}
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
|
@ -23,18 +27,23 @@ jobs:
|
|||
check-common-ancestor:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
outputs:
|
||||
review_status: ${{ steps.merge-base-check.outputs.review_status }}
|
||||
steps:
|
||||
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
fetch-depth: 0 # full history both sides for merge-base
|
||||
|
||||
- name: Reject PRs with no common ancestor on main
|
||||
- id: merge-base-check
|
||||
name: Reject PRs with no common ancestor on main
|
||||
run: |
|
||||
# `git merge-base` exits non-zero AND prints nothing when the two
|
||||
# commits share no ancestor. We check both conditions explicitly
|
||||
# so the failure message is clear regardless of which signal fires
|
||||
# first.
|
||||
if ! BASE=$(git merge-base origin/main HEAD 2>/dev/null) || [ -z "$BASE" ]; then
|
||||
STATUS='[{"source":"unrelated histories","results":[{"kind":"action_required","title":"Unrelated histories","summary":"This PR has no common ancestor with main.","detail":"","how_to_fix":"Rebase your changes onto current main:\n```\ngit fetch origin main\ngit checkout -b fix-branch origin/main\n# re-apply your changes (cherry-pick, copy files, etc.)\ngit push -f origin fix-branch\n```\n"}]}]'
|
||||
echo "review_status=${STATUS}" >> "$GITHUB_OUTPUT"
|
||||
echo ""
|
||||
echo "::error::This PR has no common ancestor with main."
|
||||
echo ""
|
||||
|
|
@ -56,3 +65,4 @@ jobs:
|
|||
exit 1
|
||||
fi
|
||||
echo "::notice::Common ancestor with main: $BASE"
|
||||
echo "review_status=[]" >> "$GITHUB_OUTPUT"
|
||||
|
|
|
|||
81
.github/workflows/label-rerun.yml
vendored
Normal file
81
.github/workflows/label-rerun.yml
vendored
Normal file
|
|
@ -0,0 +1,81 @@
|
|||
name: Label rerun
|
||||
|
||||
# When the ``ci-reviewed`` label is added to a PR, rerun all failed jobs in
|
||||
# the latest CI run. This re-evaluates ``review-labels`` (which now sees the
|
||||
# label) and GitHub automatically reruns dependent jobs (``comment-live``,
|
||||
# ``all-checks-pass``) — so the review comment gets updated too.
|
||||
#
|
||||
# If the CI run is still in progress when the label is added, we wait for it
|
||||
# to finish before rerunning (``gh run rerun`` only works on completed runs).
|
||||
# The wait can be long (20+ min for a full CI run), but it's better than
|
||||
# silently failing and leaving the reviewer stuck.
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
types: [labeled]
|
||||
|
||||
permissions:
|
||||
actions: write
|
||||
pull-requests: read
|
||||
|
||||
concurrency:
|
||||
group: label-rerun-${{ github.event.pull_request.number }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
rerun-review-labels:
|
||||
name: Rerun review-labels job
|
||||
if: github.event.label.name == 'ci-reviewed'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 40
|
||||
steps:
|
||||
- name: Wait for CI run to finish, then rerun failed jobs
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
REPO: ${{ github.repository }}
|
||||
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
||||
run: |
|
||||
set -uo pipefail
|
||||
|
||||
# Find the latest CI run for this PR's head SHA.
|
||||
RUN_ID=$(gh run list \
|
||||
--repo "$REPO" \
|
||||
--commit "$HEAD_SHA" \
|
||||
--workflow ci.yml \
|
||||
--limit 1 \
|
||||
--json databaseId,status \
|
||||
--jq '.[0] | "\(.databaseId) \(.status)"' 2>/dev/null || true)
|
||||
|
||||
if [ -z "$RUN_ID" ]; then
|
||||
echo "No CI run found for this PR — nothing to rerun."
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Split "RUN_ID STATUS" into two vars.
|
||||
RUN_ID="${RUN_ID%% *}"
|
||||
STATUS="${RUN_ID##* }"
|
||||
|
||||
echo "Latest CI run: $RUN_ID (status: $STATUS)"
|
||||
|
||||
# If the run is still in progress, wait for it to finish.
|
||||
# gh run rerun only works on completed runs — if we try while it's
|
||||
# running, GitHub rejects with "cannot be rerun; This workflow is
|
||||
# already running".
|
||||
if [ "$STATUS" != "completed" ]; then
|
||||
echo "Run is $STATUS — waiting for completion (this may take a while)..."
|
||||
# gh run watch --exit-status exits non-zero if the run fails,
|
||||
# which is expected (the label gate fails). Don't let that kill
|
||||
# the workflow — we WANT to rerun failed jobs.
|
||||
timeout 2100 gh run watch "$RUN_ID" --repo "$REPO" --interval 15 || true
|
||||
|
||||
# Verify it's actually completed now.
|
||||
STATUS=$(gh run view "$RUN_ID" --repo "$REPO" --json status --jq '.status' 2>/dev/null || echo "unknown")
|
||||
if [ "$STATUS" != "completed" ]; then
|
||||
echo "Run is still $STATUS after wait — giving up."
|
||||
exit 0
|
||||
fi
|
||||
fi
|
||||
|
||||
echo "Run completed. Rerunning all failed jobs..."
|
||||
gh run rerun "$RUN_ID" --repo "$REPO" --failed || true
|
||||
echo "Done. GitHub will rerun review-labels and all dependent jobs."
|
||||
126
.github/workflows/lint.yml
vendored
126
.github/workflows/lint.yml
vendored
|
|
@ -2,11 +2,14 @@ name: Lint (ruff + ty)
|
|||
|
||||
# Two things here:
|
||||
# 1. Advisory diff — ruff + ty diagnostics as a diff vs the target branch.
|
||||
# Posts a Markdown summary and a PR comment. Exit zero always.
|
||||
# Writes a Markdown summary to the run page. Exit zero always.
|
||||
# 2. Blocking ``ruff check .`` — enforces the explicit rules in
|
||||
# ``[tool.ruff.lint.select]`` (currently PLW1514). Failure blocks merge.
|
||||
# Separate job so the advisory diff still runs and posts even when
|
||||
# enforcement fails.
|
||||
# Separate job so the advisory diff still runs even when enforcement
|
||||
# fails.
|
||||
#
|
||||
# CI-sensitive file review was previously here as a ``ci-review`` job but
|
||||
# has moved to ``review-labels.yml`` so it can be rerun independently.
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
|
|
@ -15,14 +18,9 @@ on:
|
|||
description: The event name from the calling orchestrator (pull_request or push).
|
||||
type: string
|
||||
required: true
|
||||
ci_review:
|
||||
description: Whether CI-sensitive files (eslint config, workflows, actions) changed and require a review label.
|
||||
type: boolean
|
||||
default: false
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: write # needed to post/update PR comments
|
||||
|
||||
concurrency:
|
||||
group: lint-${{ github.ref }}
|
||||
|
|
@ -162,115 +160,3 @@ jobs:
|
|||
|
||||
- name: Run footgun checker
|
||||
run: python scripts/check-windows-footguns.py --all
|
||||
|
||||
ci-review:
|
||||
# Require explicit maintainer review when CI-sensitive files change:
|
||||
# eslint config, workflow YAMLs, or composite actions. These files
|
||||
# influence what code the js-autofix job executes and pushes to
|
||||
# main, so a malicious PR could inject arbitrary code via a custom eslint
|
||||
# rule's `fix` function. The label gate ensures a human reviews before
|
||||
# merge. Mirrors the mcp-catalog-reviewed pattern in supply-chain-audit.yml.
|
||||
name: CI-sensitive file review
|
||||
if: inputs.event_name == 'pull_request' && inputs.ci_review
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 2
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
- name: Require ci-reviewed label
|
||||
id: label-check
|
||||
env:
|
||||
# Read-only label lookup. Use the built-in GITHUB_TOKEN (present and
|
||||
# read-only on forks) so the gate works on fork PRs; fall back to it
|
||||
# when AUTOFIX_BOT_PAT is empty. `|| true` degrades an API blip to
|
||||
# "label absent" rather than hard-failing the step.
|
||||
GH_TOKEN: ${{ secrets.AUTOFIX_BOT_PAT || github.token }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
PR="${{ github.event.pull_request.number }}"
|
||||
LABELS=$(gh pr view "$PR" --json labels --jq '.labels[].name' || true)
|
||||
if echo "$LABELS" | grep -Fxq 'ci-reviewed'; then
|
||||
echo "reviewed=true" >> "$GITHUB_OUTPUT"
|
||||
echo "ci-reviewed label present."
|
||||
exit 0
|
||||
fi
|
||||
echo "reviewed=false" >> "$GITHUB_OUTPUT"
|
||||
|
||||
# On failure: find the bot's previous comment and edit it, or create
|
||||
# a new one if none exists. Using an HTML comment marker so we can
|
||||
# locate it reliably across runs without parsing the body text.
|
||||
# Skipped on fork PRs — GITHUB_TOKEN is read-only there, so the API
|
||||
# call would fail. The label gate still holds via the step below.
|
||||
- name: Post or update review warning
|
||||
if: steps.label-check.outputs.reviewed != 'true' && github.event.pull_request.head.repo.fork != true
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.AUTOFIX_BOT_PAT || github.token }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
PR="${{ github.event.pull_request.number }}"
|
||||
MARKER="<!-- ci-review-bot -->"
|
||||
BODY="${MARKER}
|
||||
## ⚠️ CI-sensitive file review required
|
||||
|
||||
This PR changes CI-sensitive files (eslint config, workflow YAMLs,
|
||||
or composite actions). These files influence what code the
|
||||
js-autofix job executes and pushes to main.
|
||||
|
||||
A maintainer should verify:
|
||||
- no new eslint rules with custom \`fix\` functions that write outside linted paths,
|
||||
- no workflow changes that widen permissions or remove guards,
|
||||
- no composite action changes that alter what gets executed.
|
||||
|
||||
After review, add the \`ci-reviewed\` label and re-run this check."
|
||||
|
||||
# Find an existing comment with our marker.
|
||||
COMMENT_ID=$(gh api \
|
||||
"repos/${{ github.repository }}/issues/${PR}/comments" \
|
||||
--paginate --jq ".[] | select(.body | contains(\"${MARKER}\")) | .id" \
|
||||
| head -1 || true)
|
||||
|
||||
if [ -n "$COMMENT_ID" ]; then
|
||||
gh api --method PATCH \
|
||||
"repos/${{ github.repository }}/issues/comments/${COMMENT_ID}" \
|
||||
-f body="$BODY"
|
||||
else
|
||||
gh pr comment "$PR" --body "$BODY"
|
||||
fi
|
||||
|
||||
# Fail the job when the label is missing — always runs (including
|
||||
# fork PRs) so the security gate holds even when the comment step
|
||||
# was skipped above.
|
||||
- name: Fail on missing label
|
||||
if: steps.label-check.outputs.reviewed != 'true'
|
||||
run: |
|
||||
echo "::error::CI-sensitive changes require the ci-reviewed label."
|
||||
exit 1
|
||||
|
||||
# On success: if a previous warning comment exists, edit it to show
|
||||
# the review passed so the PR doesn't have a stale ⚠️ sitting around.
|
||||
# Skipped on fork PRs — no comment was ever posted to update.
|
||||
- name: Update previous warning to passed
|
||||
if: steps.label-check.outputs.reviewed == 'true' && github.event.pull_request.head.repo.fork != true
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.AUTOFIX_BOT_PAT || github.token }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
PR="${{ github.event.pull_request.number }}"
|
||||
MARKER="<!-- ci-review-bot -->"
|
||||
|
||||
# Find an existing comment with our marker.
|
||||
COMMENT_ID=$(gh api \
|
||||
"repos/${{ github.repository }}/issues/${PR}/comments" \
|
||||
--paginate --jq ".[] | select(.body | contains(\"${MARKER}\")) | .id" \
|
||||
| head -1 || true)
|
||||
|
||||
if [ -n "$COMMENT_ID" ]; then
|
||||
BODY="${MARKER}
|
||||
## ✅ CI-sensitive file review passed
|
||||
|
||||
The \`ci-reviewed\` label is present on this PR."
|
||||
|
||||
gh api --method PATCH \
|
||||
"repos/${{ github.repository }}/issues/comments/${COMMENT_ID}" \
|
||||
-f body="$BODY"
|
||||
fi
|
||||
|
|
|
|||
81
.github/workflows/lockfile-diff.yml
vendored
81
.github/workflows/lockfile-diff.yml
vendored
|
|
@ -7,22 +7,25 @@ name: Lockfile diff
|
|||
# the ``packages`` map at the merge base and at HEAD and set-diffs the
|
||||
# {install path: version} maps instead.
|
||||
#
|
||||
# The comment is upserted: the script embeds a hidden HTML marker and the
|
||||
# workflow PATCHes the existing comment when one is found, so a PR gets
|
||||
# exactly one lockfile-diff comment that tracks the latest push instead
|
||||
# of a stack of stale ones. When a later push reverts all lockfile
|
||||
# changes, the comment is updated to say so (deleting it would be more
|
||||
# surprising than telling the reviewer it's resolved).
|
||||
# The semantic diff is exposed as a workflow_call output ``review_status``
|
||||
# (a JSON array in the unified status format) and an artifact
|
||||
# (``lockfile-diff`` containing the markdown fragment) for the step
|
||||
# summary.
|
||||
#
|
||||
# Never blocking — this is review signal, not enforcement. Exit is 0 even
|
||||
# when commenting fails (fork PRs get a read-only GITHUB_TOKEN).
|
||||
# Never blocking — this is review signal, not enforcement.
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
outputs:
|
||||
changed:
|
||||
description: Whether package-lock.json changed relative to the target branch.
|
||||
value: ${{ jobs.diff.outputs.changed }}
|
||||
review_status:
|
||||
description: JSON array of review status objects for the unified PR comment.
|
||||
value: ${{ jobs.diff.outputs.review_status }}
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: write # post/update the diff comment
|
||||
|
||||
concurrency:
|
||||
group: lockfile-diff-${{ github.event.pull_request.number || github.ref }}
|
||||
|
|
@ -33,6 +36,9 @@ jobs:
|
|||
name: package-lock.json semantic diff
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 5
|
||||
outputs:
|
||||
changed: ${{ steps.diff.outputs.changed }}
|
||||
review_status: ${{ steps.emit-status.outputs.review_status }}
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
|
|
@ -54,45 +60,36 @@ jobs:
|
|||
--output /tmp/lockfile-diff.md
|
||||
if [ -s /tmp/lockfile-diff.md ]; then
|
||||
echo "changed=true" >> "$GITHUB_OUTPUT"
|
||||
cat /tmp/lockfile-diff.md >> "$GITHUB_STEP_SUMMARY"
|
||||
{
|
||||
echo "## package-lock.json semantic diff"
|
||||
echo ""
|
||||
cat /tmp/lockfile-diff.md
|
||||
} >> "$GITHUB_STEP_SUMMARY"
|
||||
else
|
||||
echo "changed=false" >> "$GITHUB_OUTPUT"
|
||||
: > /tmp/lockfile-diff.md
|
||||
fi
|
||||
|
||||
- name: Post or update PR comment
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.AUTOFIX_BOT_PAT }}
|
||||
REPO: ${{ github.repository }}
|
||||
PR: ${{ github.event.pull_request.number }}
|
||||
CHANGED: ${{ steps.diff.outputs.changed }}
|
||||
- name: Emit review_status
|
||||
id: emit-status
|
||||
run: |
|
||||
set -euo pipefail
|
||||
MARKER='<!-- hermes-lockfile-diff -->'
|
||||
CHANGED="${{ steps.diff.outputs.changed }}"
|
||||
STATUS="[]"
|
||||
|
||||
# Find our previous comment (paginated — busy PRs exceed one page).
|
||||
EXISTING=$(gh api --paginate "repos/${REPO}/issues/${PR}/comments" \
|
||||
--jq ".[] | select(.body | startswith(\"$MARKER\")) | .id" \
|
||||
| head -1 || true)
|
||||
|
||||
if [ "$CHANGED" != "true" ]; then
|
||||
if [ -n "$EXISTING" ]; then
|
||||
# A previous push changed the lockfile but the latest one
|
||||
# doesn't — update the comment rather than leave stale info.
|
||||
printf '%s\n✅ package-lock.json changes from an earlier push have been reverted — locked versions now match the target branch.\n' "$MARKER" > /tmp/lockfile-diff.md
|
||||
else
|
||||
echo "No lockfile changes and no existing comment — nothing to do."
|
||||
exit 0
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ -n "$EXISTING" ]; then
|
||||
echo "Updating existing comment ${EXISTING}"
|
||||
gh api --method PATCH "repos/${REPO}/issues/comments/${EXISTING}" \
|
||||
-F body=@/tmp/lockfile-diff.md > /dev/null \
|
||||
|| echo "::warning::Could not update PR comment (expected for fork PRs — GITHUB_TOKEN is read-only)"
|
||||
if [ "$CHANGED" = "true" ]; then
|
||||
CONTENT=$(cat /tmp/lockfile-diff.md | python3 -c "import sys,json; print(json.dumps(sys.stdin.read()))")
|
||||
STATUS="[{\"source\":\"lockfile-diff\",\"results\":[{\"kind\":\"action_required\",\"title\":\"package-lock.json\",\"summary\":\"Locked npm dependency versions changed.\",\"detail\":${CONTENT},\"how_to_fix\":\"Add the \`ci-reviewed\` label after verifying the version changes are expected.\"}]}"
|
||||
else
|
||||
echo "Creating new comment"
|
||||
gh api "repos/${REPO}/issues/${PR}/comments" \
|
||||
-F body=@/tmp/lockfile-diff.md > /dev/null \
|
||||
|| echo "::warning::Could not post PR comment (expected for fork PRs — GITHUB_TOKEN is read-only)"
|
||||
STATUS="[]"
|
||||
fi
|
||||
|
||||
echo "review_status=${STATUS}" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Upload diff artifact
|
||||
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
||||
with:
|
||||
name: lockfile-diff
|
||||
path: /tmp/lockfile-diff.md
|
||||
retention-days: 1
|
||||
overwrite: true
|
||||
|
|
|
|||
139
.github/workflows/osv-scanner.yml
vendored
139
.github/workflows/osv-scanner.yml
vendored
|
|
@ -14,14 +14,14 @@ name: OSV-Scanner
|
|||
# code patterns in PR diffs) by covering the orthogonal "currently-pinned
|
||||
# dep became known-vulnerable" case.
|
||||
#
|
||||
# Steps below are inlined from Google's officially-recommended reusable
|
||||
# workflow (google/osv-scanner-action/.github/workflows/osv-scanner-reusable.yml),
|
||||
# rather than called via `uses:` so we can set a `timeout-minutes` in the
|
||||
# degenerate case where this job hangs.
|
||||
|
||||
# Uses Google's officially-recommended reusable workflow, pinned by SHA.
|
||||
# Findings land in the repo's Security tab (Code Scanning > OSV-Scanner).
|
||||
# fail-on-vuln is disabled so the job does not block merges on pre-existing
|
||||
# vulnerabilities in pinned deps that we may need to patch deliberately.
|
||||
#
|
||||
# The reusable workflow can't emit custom outputs, so a wrapper job
|
||||
# downloads the SARIF result and summarizes the vulnerability count into
|
||||
# a review_status for the unified PR comment.
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
|
|
@ -40,62 +40,85 @@ permissions:
|
|||
jobs:
|
||||
scan:
|
||||
name: Scan lockfiles
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
uses: google/osv-scanner-action/.github/workflows/osv-scanner-reusable.yml@9a498708959aeaef5ef730655706c5a1df1edbc2 # v2.3.8
|
||||
with:
|
||||
# Scan explicit lockfiles rather than recursing, so we only look at
|
||||
# the three sources of truth and skip vendored / test / worktree dirs.
|
||||
scan-args: |-
|
||||
--lockfile=uv.lock
|
||||
--lockfile=package-lock.json
|
||||
--lockfile=website/package-lock.json
|
||||
fail-on-vuln: false
|
||||
|
||||
- name: 'Run scanner'
|
||||
uses: google/osv-scanner-action/osv-scanner-action@9a498708959aeaef5ef730655706c5a1df1edbc2 # v2.3.8
|
||||
emit-status:
|
||||
name: Emit review status
|
||||
runs-on: ubuntu-latest
|
||||
needs: scan
|
||||
if: always()
|
||||
outputs:
|
||||
review_status: ${{ steps.emit.outputs.review_status }}
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
|
||||
- name: Download SARIF result
|
||||
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
|
||||
with:
|
||||
# Scan explicit lockfiles rather than recursing, so we only look at
|
||||
# the three sources of truth and skip vendored / test / worktree dirs.
|
||||
scan-args: |-
|
||||
--output=results.json
|
||||
--format=json
|
||||
--lockfile=uv.lock
|
||||
--lockfile=package-lock.json
|
||||
--lockfile=website/package-lock.json
|
||||
name: osv-results
|
||||
path: /tmp/osv-results
|
||||
continue-on-error: true
|
||||
|
||||
- name: 'Run osv-scanner-reporter'
|
||||
uses: google/osv-scanner-action/osv-reporter-action@9a498708959aeaef5ef730655706c5a1df1edbc2 # v2.3.8
|
||||
with:
|
||||
scan-args: |-
|
||||
--output=results.sarif
|
||||
--new=results.json
|
||||
--gh-annotations=false
|
||||
--fail-on-vuln=false
|
||||
|
||||
# Upload the results as artifacts (optional). Commenting out will disable uploads of run results in SARIF
|
||||
# format to the repository Actions tab.
|
||||
- name: 'Upload artifact'
|
||||
id: 'upload_artifact'
|
||||
if: ${{ !cancelled() }}
|
||||
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
|
||||
with:
|
||||
name: OSV Scanner SARIF file
|
||||
path: results.sarif
|
||||
retention-days: 5
|
||||
|
||||
# Upload the results to GitHub's code scanning dashboard.
|
||||
- name: 'Upload to code-scanning'
|
||||
if: ${{ !cancelled() }}
|
||||
uses: github/codeql-action/upload-sarif@cdefb33c0f6224e58673d9004f47f7cb3e328b89 # v4.31.10
|
||||
with:
|
||||
sarif_file: results.sarif
|
||||
|
||||
- name: 'Print Code Scanning URL'
|
||||
if: ${{ !cancelled() }}
|
||||
- name: Emit review_status
|
||||
id: emit
|
||||
run: |
|
||||
echo "View the OSV-Scanner results in the 'Security' tab, using the following link:"
|
||||
echo "${{ github.server_url }}/${{ github.repository }}/security/code-scanning?query=is%3Aopen+branch%3A${GITHUB_REF_NAME}+tool%3Aosv-scanner"
|
||||
env:
|
||||
GITHUB_REF_NAME: ${{ github.ref_name }}
|
||||
set -euo pipefail
|
||||
STATUS="[]"
|
||||
|
||||
- name: 'Error troubleshooter'
|
||||
if: ${{ always() && steps.upload_artifact.outcome == 'failure' }}
|
||||
run: |
|
||||
echo "::error::Artifact upload failed. This is most likely caused by a error during scanning earlier in the workflow."
|
||||
exit 1
|
||||
if [ -f /tmp/osv-results/osv-results.sarif ]; then
|
||||
# Count vulnerabilities from the SARIF file
|
||||
VULN_COUNT=$(python3 -c "
|
||||
import json, sys
|
||||
try:
|
||||
with open('/tmp/osv-results/osv-results.sarif') as f:
|
||||
data = json.load(f)
|
||||
count = 0
|
||||
vulns = []
|
||||
for run in data.get('runs', []):
|
||||
for result in run.get('results', []):
|
||||
count += 1
|
||||
rule_id = result.get('ruleId', 'unknown')
|
||||
message = result.get('message', {}).get('text', '')
|
||||
loc = result.get('locations', [{}])[0].get('physicalLocation', {}).get('artifactLocation', {}).get('uri', '')
|
||||
vulns.append(f'- {rule_id} in {loc}: {message}')
|
||||
print(count)
|
||||
if vulns:
|
||||
print('\n'.join(vulns[:20]), file=sys.stderr)
|
||||
except Exception:
|
||||
print(0)
|
||||
")
|
||||
|
||||
VULN_DETAIL=""
|
||||
if [ "$VULN_COUNT" -gt 0 ] 2>/dev/null; then
|
||||
VULN_PLURAL=$([ "$VULN_COUNT" -eq 1 ] && echo "y" || echo "ies")
|
||||
VULN_DETAIL=$(python3 -c "
|
||||
import json, sys
|
||||
try:
|
||||
with open('/tmp/osv-results/osv-results.sarif') as f:
|
||||
data = json.load(f)
|
||||
vulns = []
|
||||
for run in data.get('runs', []):
|
||||
for result in run.get('results', []):
|
||||
rule_id = result.get('ruleId', 'unknown')
|
||||
loc = result.get('locations', [{}])[0].get('physicalLocation', {}).get('artifactLocation', {}).get('uri', '')
|
||||
vulns.append(f'- {rule_id} in {loc}')
|
||||
print(json.dumps('\n'.join(vulns[:20])))
|
||||
except Exception:
|
||||
print(json.dumps(''))
|
||||
")
|
||||
STATUS="[{\"source\":\"osv scan\",\"results\":[{\"kind\":\"warning\",\"title\":\"OSV vulnerability scan\",\"summary\":\"${VULN_COUNT} known vulnerabilit${VULN_PLURAL} found in pinned dependencies.\",\"detail\":${VULN_DETAIL},\"how_to_fix\":\"Review the findings in the [Security tab](../../security/code-scanning). Update the affected dependencies if a patched version is available.\"}]}]"
|
||||
else
|
||||
STATUS="[]"
|
||||
fi
|
||||
fi
|
||||
|
||||
echo "review_status=${STATUS}" >> "$GITHUB_OUTPUT"
|
||||
|
|
|
|||
92
.github/workflows/review-labels.yml
vendored
Normal file
92
.github/workflows/review-labels.yml
vendored
Normal file
|
|
@ -0,0 +1,92 @@
|
|||
name: Review labels
|
||||
|
||||
# Require explicit maintainer review when CI-sensitive files or the MCP
|
||||
# catalog change. Previously this was split across two jobs in two
|
||||
# workflows: ``ci-review`` in lint.yml (gated on ``ci_review``) and
|
||||
# ``mcp-catalog-review`` in supply-chain-audit.yml (gated on
|
||||
# ``mcp_catalog``). Both checked for their own label.
|
||||
#
|
||||
# Now consolidated: a single ``ci-reviewed`` label covers both. The
|
||||
# comment sections tell the reviewer exactly what to verify per area,
|
||||
# so one label is enough — the human reads the comment, not the label
|
||||
# name.
|
||||
#
|
||||
# Outputs:
|
||||
# ci_reviewed — "true" / "false" / "" (empty when neither lane ran)
|
||||
# review_status — JSON array of status objects consumed by the review
|
||||
# comment assembler. See scripts/ci/emit_review_status.py.
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
ci_review:
|
||||
description: Whether CI-sensitive files (eslint config, workflows, actions) changed.
|
||||
type: boolean
|
||||
default: false
|
||||
mcp_catalog:
|
||||
description: Whether the MCP catalog / installer changed.
|
||||
type: boolean
|
||||
default: false
|
||||
outputs:
|
||||
ci_reviewed:
|
||||
description: Whether the ci-reviewed label is present. Empty when neither input was true.
|
||||
value: ${{ jobs.check.outputs.ci_reviewed }}
|
||||
review_status:
|
||||
description: JSON array of status objects for the review comment assembler.
|
||||
value: ${{ jobs.check.outputs.review_status }}
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: read # read PR labels
|
||||
|
||||
jobs:
|
||||
check:
|
||||
name: Review label gate
|
||||
if: inputs.ci_review || inputs.mcp_catalog
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 2
|
||||
outputs:
|
||||
ci_reviewed: ${{ steps.label-check.outputs.ci_reviewed }}
|
||||
review_status: ${{ steps.build-status.outputs.review_status }}
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
|
||||
- name: Check ci-reviewed label
|
||||
id: label-check
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
REPO: ${{ github.repository }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
PR="${{ github.event.pull_request.number }}"
|
||||
LABELS=$(gh pr view "$PR" --repo "$REPO" --json labels --jq '.labels[].name' || true)
|
||||
|
||||
if echo "$LABELS" | grep -Fxq 'ci-reviewed'; then
|
||||
echo "ci-reviewed label present."
|
||||
echo "ci_reviewed=true" >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
echo "ci-reviewed label missing."
|
||||
echo "ci_reviewed=false" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
|
||||
- name: Build review_status JSON
|
||||
id: build-status
|
||||
env:
|
||||
CI_REVIEW: ${{ inputs.ci_review }}
|
||||
MCP_CATALOG: ${{ inputs.mcp_catalog }}
|
||||
LABEL_PRESENT: ${{ steps.label-check.outputs.ci_reviewed }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
ARGS=""
|
||||
if [ "$CI_REVIEW" = "true" ]; then ARGS="$ARGS --ci-review"; fi
|
||||
if [ "$MCP_CATALOG" = "true" ]; then ARGS="$ARGS --mcp-catalog"; fi
|
||||
if [ "$LABEL_PRESENT" = "true" ]; then ARGS="$ARGS --label-present"; fi
|
||||
|
||||
python3 scripts/ci/emit_review_status.py $ARGS --output "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Fail on missing label
|
||||
if: steps.label-check.outputs.ci_reviewed != 'true'
|
||||
run: |
|
||||
echo "::error::CI-sensitive changes require the ci-reviewed label. Add the label and re-run this check."
|
||||
exit 1
|
||||
168
.github/workflows/supply-chain-audit.yml
vendored
168
.github/workflows/supply-chain-audit.yml
vendored
|
|
@ -10,9 +10,17 @@ name: Supply Chain Audit
|
|||
# advisory-only workflow instead.
|
||||
#
|
||||
# Path-gating is handled centrally by the ``ci.yml`` orchestrator's
|
||||
# ``detect`` job. The orchestrator passes ``scan`` / ``deps`` /
|
||||
# ``mcp_catalog`` booleans as inputs; this workflow's jobs gate on those
|
||||
# inputs instead of re-computing the diff.
|
||||
# ``detect`` job. The orchestrator passes ``scan`` / ``deps`` booleans as
|
||||
# inputs; this workflow's jobs gate on those inputs instead of re-computing
|
||||
# the diff. MCP catalog review was previously here but has moved to
|
||||
# ``review-labels.yml`` so it can be rerun independently.
|
||||
#
|
||||
# Outputs:
|
||||
# review_status — JSON array of status objects consumed by the review
|
||||
# comment assembler (scripts/ci/assemble_review_comment.py).
|
||||
# Each job (``scan``, ``dep-bounds``) emits its own
|
||||
# array; an ``aggregate`` job merges them into the
|
||||
# workflow-level output.
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
|
|
@ -29,10 +37,10 @@ on:
|
|||
description: Whether pyproject.toml changed.
|
||||
type: boolean
|
||||
required: true
|
||||
mcp_catalog:
|
||||
description: Whether the MCP catalog / installer changed.
|
||||
type: boolean
|
||||
required: true
|
||||
outputs:
|
||||
review_status:
|
||||
description: JSON array of review status objects for the review comment assembler.
|
||||
value: ${{ jobs.aggregate.outputs.review_status }}
|
||||
|
||||
permissions:
|
||||
pull-requests: write
|
||||
|
|
@ -44,6 +52,8 @@ jobs:
|
|||
if: inputs.scan
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 15
|
||||
outputs:
|
||||
review_status: ${{ steps.emit-status.outputs.review_status }}
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
|
|
@ -61,7 +71,7 @@ jobs:
|
|||
HEAD="${{ github.event.pull_request.head.sha }}"
|
||||
|
||||
# Added lines only, excluding lockfiles.
|
||||
# Three-dot diff (base...head) diffs from the merge base to HEAD,
|
||||
# Three-point diff (base...head) diffs from the merge base to HEAD,
|
||||
# so only changes introduced by this PR are included — not changes
|
||||
# that landed on main after the PR branched off.
|
||||
DIFF=$(git diff "$BASE"..."$HEAD" -- . ':!uv.lock' ':!*.lock' ':!package-lock.json' ':!yarn.lock' || true)
|
||||
|
|
@ -139,26 +149,41 @@ jobs:
|
|||
echo "found=false" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
|
||||
- name: Post critical finding comment
|
||||
if: steps.scan.outputs.found == 'true'
|
||||
- name: Emit review_status
|
||||
id: emit-status
|
||||
if: always()
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.AUTOFIX_BOT_PAT }}
|
||||
FOUND: ${{ steps.scan.outputs.found }}
|
||||
run: |
|
||||
BODY="## 🚨 CRITICAL Supply Chain Risk Detected
|
||||
python3 - <<'PYEOF'
|
||||
import json, os
|
||||
|
||||
This PR contains a pattern that has been used in real supply chain attacks. A maintainer must review the flagged code carefully before merging.
|
||||
found = os.environ.get("FOUND", "") == "true"
|
||||
|
||||
$(cat /tmp/findings.md)
|
||||
if found:
|
||||
with open("/tmp/findings.md", encoding="utf-8") as f:
|
||||
detail = f.read()
|
||||
status = [{
|
||||
"source": "supply chain",
|
||||
"results": [{
|
||||
"kind": "error",
|
||||
"title": "Critical supply chain risk",
|
||||
"summary": "Critical supply chain risk patterns detected in this PR.",
|
||||
"detail": detail,
|
||||
"how_to_fix": "Review the flagged code carefully. If intentional, add the `ci-reviewed` label."
|
||||
}]
|
||||
}]
|
||||
else:
|
||||
status = []
|
||||
|
||||
---
|
||||
*Scanner only fires on high-signal indicators: .pth files, base64+exec/eval combos, subprocess with encoded commands, or install-hook files. Low-signal warnings were removed intentionally — if you're seeing this comment, the finding is worth inspecting.*"
|
||||
|
||||
gh pr comment "${{ github.event.pull_request.number }}" --body "$BODY" || echo "::warning::Could not post PR comment (expected for fork PRs — GITHUB_TOKEN is read-only)"
|
||||
with open(os.environ["GITHUB_OUTPUT"], "a", encoding="utf-8") as f:
|
||||
f.write(f"review_status={json.dumps(status)}\n")
|
||||
PYEOF
|
||||
|
||||
- name: Fail on critical findings
|
||||
if: steps.scan.outputs.found == 'true'
|
||||
run: |
|
||||
echo "::error::CRITICAL supply chain risk patterns detected in this PR. See the PR comment for details."
|
||||
echo "::error::CRITICAL supply chain risk patterns detected in this PR. See the review comment for details."
|
||||
exit 1
|
||||
|
||||
dep-bounds:
|
||||
|
|
@ -166,6 +191,8 @@ jobs:
|
|||
if: inputs.deps
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 15
|
||||
outputs:
|
||||
review_status: ${{ steps.emit-status.outputs.review_status }}
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
|
|
@ -188,7 +215,7 @@ jobs:
|
|||
exit 0
|
||||
fi
|
||||
|
||||
# Match PyPI dep specs that have >= but no < ceiling.
|
||||
# Match PyPI dep specs that have >= and no < ceiling.
|
||||
# Pattern: "package>=version" without a following ",<" bound.
|
||||
# Excludes git+ URLs (which use commit SHAs) and comments.
|
||||
UNBOUNDED=$(echo "$ADDED" | grep -oE '"[a-zA-Z0-9_-]+(\[[^\]]*\])?>=[ 0-9.]+"' | grep -v ',<' || true)
|
||||
|
|
@ -200,26 +227,36 @@ jobs:
|
|||
echo "found=false" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
|
||||
- name: Post unbounded dep warning
|
||||
if: steps.bounds.outputs.found == 'true'
|
||||
- name: Emit review_status
|
||||
id: emit-status
|
||||
if: always()
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.AUTOFIX_BOT_PAT }}
|
||||
FOUND: ${{ steps.bounds.outputs.found }}
|
||||
run: |
|
||||
BODY="## ⚠️ Unbounded PyPI Dependency Detected
|
||||
python3 - <<'PYEOF'
|
||||
import json, os
|
||||
|
||||
This PR adds PyPI dependencies without a \`<next_major\` upper bound. Per our [supply chain policy](../blob/main/CONTRIBUTING.md#dependency-pinning-policy-supply-chain-hardening), all PyPI deps must be pinned as \`>=floor,<next_major\`.
|
||||
found = os.environ.get("FOUND", "") == "true"
|
||||
|
||||
**Unbounded specs found:**
|
||||
\`\`\`
|
||||
$(cat /tmp/unbounded.txt)
|
||||
\`\`\`
|
||||
if found:
|
||||
with open("/tmp/unbounded.txt", encoding="utf-8") as f:
|
||||
detail = f.read()
|
||||
status = [{
|
||||
"source": "supply chain",
|
||||
"results": [{
|
||||
"kind": "action_required",
|
||||
"title": "Unbounded PyPI dependencies",
|
||||
"summary": "This PR adds PyPI dependencies without upper bounds.",
|
||||
"detail": detail,
|
||||
"how_to_fix": 'Add a `<next_major` upper bound, e.g. `"package>=1.2.0,<2"`. See CONTRIBUTING.md dependency pinning policy.'
|
||||
}]
|
||||
}]
|
||||
else:
|
||||
status = []
|
||||
|
||||
**Fix:** Add an upper bound, e.g. \`"package>=1.2.0,<2"\`
|
||||
|
||||
---
|
||||
*See PR #2810 and CONTRIBUTING.md for the full policy rationale.*"
|
||||
|
||||
gh pr comment "${{ github.event.pull_request.number }}" --body "$BODY" || echo "::warning::Could not post PR comment (expected for fork PRs)"
|
||||
with open(os.environ["GITHUB_OUTPUT"], "a", encoding="utf-8") as f:
|
||||
f.write(f"review_status={json.dumps(status)}\n")
|
||||
PYEOF
|
||||
|
||||
- name: Fail on unbounded deps
|
||||
if: steps.bounds.outputs.found == 'true'
|
||||
|
|
@ -227,45 +264,36 @@ jobs:
|
|||
echo "::error::PyPI dependencies without upper bounds detected. Add <next_major ceiling per CONTRIBUTING.md policy."
|
||||
exit 1
|
||||
|
||||
mcp-catalog-review:
|
||||
name: MCP catalog security review
|
||||
if: inputs.mcp_catalog
|
||||
aggregate:
|
||||
name: Aggregate review statuses
|
||||
needs: [scan, dep-bounds]
|
||||
if: always()
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 15
|
||||
outputs:
|
||||
review_status: ${{ steps.merge.outputs.review_status }}
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Require explicit MCP catalog review label
|
||||
- name: Merge review statuses
|
||||
id: merge
|
||||
env:
|
||||
# Read-only label lookup. Use the built-in GITHUB_TOKEN (present and
|
||||
# read-only on forks) so the gate works on fork PRs; fall back to it
|
||||
# when AUTOFIX_BOT_PAT is empty. `|| true` degrades an API blip to
|
||||
# "label absent" rather than hard-failing the step.
|
||||
GH_TOKEN: ${{ secrets.AUTOFIX_BOT_PAT || github.token }}
|
||||
SCAN_STATUS: ${{ needs.scan.outputs.review_status }}
|
||||
DEP_STATUS: ${{ needs.dep-bounds.outputs.review_status }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
PR="${{ github.event.pull_request.number }}"
|
||||
LABELS=$(gh pr view "$PR" --json labels --jq '.labels[].name' || true)
|
||||
if echo "$LABELS" | grep -Fxq 'mcp-catalog-reviewed'; then
|
||||
echo "MCP catalog review label present."
|
||||
exit 0
|
||||
fi
|
||||
python3 - <<'PYEOF'
|
||||
import json, os
|
||||
|
||||
BODY="## ⚠️ MCP catalog security review required
|
||||
merged = []
|
||||
for key in ("SCAN_STATUS", "DEP_STATUS"):
|
||||
raw = os.environ.get(key, "")
|
||||
if not raw:
|
||||
continue
|
||||
try:
|
||||
data = json.loads(raw)
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
continue
|
||||
if isinstance(data, list):
|
||||
merged.extend(data)
|
||||
|
||||
This PR changes the bundled MCP catalog or MCP catalog installer code. MCP entries can define local commands that users later install into \`mcp_servers\`, so this needs explicit maintainer review before merge.
|
||||
|
||||
A maintainer should verify:
|
||||
- any new/changed \`optional-mcps/**/manifest.yaml\` command and args are expected,
|
||||
- stdio transports do not use shell+egress/exfiltration payloads,
|
||||
- git install refs are pinned and bootstrap commands are minimal,
|
||||
- requested env vars/secrets match the upstream MCP's documented needs.
|
||||
|
||||
After review, add the \`mcp-catalog-reviewed\` label and re-run this check."
|
||||
|
||||
gh pr comment "$PR" --body "$BODY" || echo "::warning::Could not post PR comment (expected for fork PRs)"
|
||||
echo "::error::MCP catalog changes require the mcp-catalog-reviewed label."
|
||||
exit 1
|
||||
with open(os.environ["GITHUB_OUTPUT"], "a", encoding="utf-8") as f:
|
||||
f.write(f"review_status={json.dumps(merged)}\n")
|
||||
PYEOF
|
||||
|
|
|
|||
11
.github/workflows/uv-lockfile-check.yml
vendored
11
.github/workflows/uv-lockfile-check.yml
vendored
|
|
@ -45,6 +45,10 @@ name: uv.lock check
|
|||
|
||||
on:
|
||||
workflow_call:
|
||||
outputs:
|
||||
review_status:
|
||||
description: "JSON review status for the review-status aggregator"
|
||||
value: ${{ jobs.check.outputs.review_status }}
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
|
@ -58,6 +62,8 @@ jobs:
|
|||
name: uv lock --check
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 5
|
||||
outputs:
|
||||
review_status: ${{ steps.verify.outputs.review_status }}
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
|
|
@ -73,6 +79,7 @@ jobs:
|
|||
# of this file) — failures often mean "your branch is behind main,
|
||||
# rebase and regenerate uv.lock."
|
||||
- name: Verify uv.lock is up-to-date
|
||||
id: verify
|
||||
run: |
|
||||
# uv lock --check re-resolves against PyPI (network). Retry so a
|
||||
# registry blip doesn't read as "lockfile stale". A genuinely stale
|
||||
|
|
@ -117,5 +124,9 @@ jobs:
|
|||
on `main` post-merge.
|
||||
EOF
|
||||
echo "::error title=uv.lock out of sync::Run \`uv lock\` locally and commit the result. If on a PR, sync with main first."
|
||||
review_status='[{"source":"uv.lock check","results":[{"kind":"action_required","title":"uv.lock out of sync","summary":"uv.lock is out of sync with pyproject.toml.","how_to_fix":"Run `uv lock` locally and commit the result. If on a PR, sync with main first:\n```\ngit fetch origin main\ngit rebase origin/main\nuv lock\ngit add uv.lock\ngit commit -m \"chore: refresh uv.lock\"\n```\n"}]}]'
|
||||
echo "review_status=${review_status}" >> "$GITHUB_OUTPUT"
|
||||
exit 1
|
||||
fi
|
||||
review_status='[]'
|
||||
echo "review_status=${review_status}" >> "$GITHUB_OUTPUT"
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue