mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-07-23 16:36:23 +00:00
Replace the static comment-pending + comment-results two-job pattern
with a live-updating comment system that polls the GitHub Actions API
every 15s, re-assembles the review comment from whatever results are
available, and upserts it via the <!-- hermes-ci-review-bot --> marker.
The comment updates in real time as each job finishes — no waiting for
the full pipeline.
Every CI job that wants to appear in the review comment emits a
review_status output — a JSON array of objects, each with a source
and a results array:
[
{
"source": "review-label-gate",
"results": [
{"kind": "action_required", "title": "...", "summary": "...",
"how_to_fix": "..."},
{"kind": "info", "title": "...", "summary": "..."}
]
},
{
"source": "ci timing",
"results": [
{"kind": "warning", "title": "CI timings", "summary": "...",
"detail": "...", "link": "..."}
]
}
]
One job can emit multiple results of different kinds. The source field
is used to exclude the corresponding job from the synthesized error
list (case-insensitive, hyphen-normalized matching against GitHub
Actions job display names).
| job | source | kind (on failure) | section |
|----------------------------|--------------------------|---------------------------|----------------------|
| review-labels | review label gate | action_required / info | Action required |
| lockfile-diff | lockfile-diff | action_required | Action required |
| ci-timings | ci timing | warning / info | Warnings |
| supply-chain scan | supply chain | error / (none) | Job failures |
| supply-chain dep-bounds | supply chain | action_required / (none) | Action required |
| osv-scanner | osv scan | warning / (none) | Warnings |
| uv-lockfile-check | uv.lock check | action_required / (none) | Action required |
| history-check | unrelated histories | action_required | Action required |
| contributor-check | contributor attribution | action_required | Action required |
Jobs that find nothing emit [] (empty array) — no noise info items.
A single comment-live job polls the GitHub Actions API every 15s,
classifies jobs into (completed, pending), assembles the comment, and
upserts it. Merges review_status outputs from all needs jobs via
toJSON(needs.*.outputs.review_status), and downloads the ci-timings
artifact when it becomes available. Shows commit SHA + message below
the header.
The assembler has ZERO job-specific knowledge. It just:
1. collect_from_statuses() — flattens all nested status objects into ReviewItems
2. collect_failed_jobs() — synthesizes errors for failed jobs with no declared status
3. _attach_job_urls() — fills in per-job log links for ALL items
4. render_comment() — groups by severity, renders with group headers
Each item shows links inline next to the title: View report (job-emitted
URL) and View job (auto-attached logs link). Each info item is its own
collapsible <details> block.
# ૮ >ﻌ< ა ci review
running on abc1234 — commit message first line
## ❌ Job failures
### {title} · [View job](url)
{summary}
## ⚠️ Action required
### {title} · [View job](url)
{summary}
**How to fix:**
{how_to_fix}
## ⚠️ Warnings
### {title} · [View report](url) · [View job](url)
{summary}
{detail}
<details><summary>{title}</summary>
{content}
</details>
Still running 3 jobs: ci-timings, docker
- test_assemble_review_comment.py (48 tests): collect_from_statuses,
collect_failed_jobs with exclude_sources, _attach_job_urls,
render_comment (group headers, inline links, commit info, per-item
details, pending footer), assemble integration
- test_live_comment.py (16 tests): classify_jobs pure function
- test_timings_report.py (10 tests): generate_review_status nested format
- test_lockfile_diff.py (6 tests)
- test_classify_changes.py (32 tests, pre-existing)
124 lines
5 KiB
YAML
124 lines
5 KiB
YAML
name: OSV-Scanner
|
|
|
|
# Scans lockfiles (uv.lock, package-lock.json) against the OSV vulnerability
|
|
# database. Runs on every PR/push (via the ci.yml orchestrator's workflow_call)
|
|
# and on a weekly schedule against main.
|
|
#
|
|
# This is detection-only — OSV-Scanner does NOT open PRs or modify pins.
|
|
# It reports known CVEs in currently-pinned dependency versions so we can
|
|
# decide when and how to patch on our own schedule. Our pinning strategy
|
|
# (full SHA / exact version) is preserved; only the notification signal
|
|
# is added.
|
|
#
|
|
# Complements the supply-chain-audit.yml workflow (which scans for malicious
|
|
# code patterns in PR diffs) by covering the orthogonal "currently-pinned
|
|
# dep became known-vulnerable" case.
|
|
#
|
|
# Uses Google's officially-recommended reusable workflow, pinned by SHA.
|
|
# Findings land in the repo's Security tab (Code Scanning > OSV-Scanner).
|
|
# fail-on-vuln is disabled so the job does not block merges on pre-existing
|
|
# vulnerabilities in pinned deps that we may need to patch deliberately.
|
|
#
|
|
# The reusable workflow can't emit custom outputs, so a wrapper job
|
|
# downloads the SARIF result and summarizes the vulnerability count into
|
|
# a review_status for the unified PR comment.
|
|
|
|
on:
|
|
workflow_call:
|
|
schedule:
|
|
# Weekly scan against main — catches CVEs published after merge for
|
|
# deps that haven't changed since.
|
|
- cron: '0 9 * * 1'
|
|
workflow_dispatch:
|
|
|
|
permissions:
|
|
# Required to upload SARIF file to CodeQL. See: https://github.com/github/codeql-action/issues/2117
|
|
actions: read
|
|
contents: read
|
|
security-events: write
|
|
|
|
jobs:
|
|
scan:
|
|
name: Scan lockfiles
|
|
uses: google/osv-scanner-action/.github/workflows/osv-scanner-reusable.yml@9a498708959aeaef5ef730655706c5a1df1edbc2 # v2.3.8
|
|
with:
|
|
# Scan explicit lockfiles rather than recursing, so we only look at
|
|
# the three sources of truth and skip vendored / test / worktree dirs.
|
|
scan-args: |-
|
|
--lockfile=uv.lock
|
|
--lockfile=package-lock.json
|
|
--lockfile=website/package-lock.json
|
|
fail-on-vuln: false
|
|
|
|
emit-status:
|
|
name: Emit review status
|
|
runs-on: ubuntu-latest
|
|
needs: scan
|
|
if: always()
|
|
outputs:
|
|
review_status: ${{ steps.emit.outputs.review_status }}
|
|
steps:
|
|
- name: Checkout code
|
|
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
|
|
|
- name: Download SARIF result
|
|
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
|
|
with:
|
|
name: osv-results
|
|
path: /tmp/osv-results
|
|
continue-on-error: true
|
|
|
|
- name: Emit review_status
|
|
id: emit
|
|
run: |
|
|
set -euo pipefail
|
|
STATUS="[]"
|
|
|
|
if [ -f /tmp/osv-results/osv-results.sarif ]; then
|
|
# Count vulnerabilities from the SARIF file
|
|
VULN_COUNT=$(python3 -c "
|
|
import json, sys
|
|
try:
|
|
with open('/tmp/osv-results/osv-results.sarif') as f:
|
|
data = json.load(f)
|
|
count = 0
|
|
vulns = []
|
|
for run in data.get('runs', []):
|
|
for result in run.get('results', []):
|
|
count += 1
|
|
rule_id = result.get('ruleId', 'unknown')
|
|
message = result.get('message', {}).get('text', '')
|
|
loc = result.get('locations', [{}])[0].get('physicalLocation', {}).get('artifactLocation', {}).get('uri', '')
|
|
vulns.append(f'- {rule_id} in {loc}: {message}')
|
|
print(count)
|
|
if vulns:
|
|
print('\n'.join(vulns[:20]), file=sys.stderr)
|
|
except Exception:
|
|
print(0)
|
|
")
|
|
|
|
VULN_DETAIL=""
|
|
if [ "$VULN_COUNT" -gt 0 ] 2>/dev/null; then
|
|
VULN_PLURAL=$([ "$VULN_COUNT" -eq 1 ] && echo "y" || echo "ies")
|
|
VULN_DETAIL=$(python3 -c "
|
|
import json, sys
|
|
try:
|
|
with open('/tmp/osv-results/osv-results.sarif') as f:
|
|
data = json.load(f)
|
|
vulns = []
|
|
for run in data.get('runs', []):
|
|
for result in run.get('results', []):
|
|
rule_id = result.get('ruleId', 'unknown')
|
|
loc = result.get('locations', [{}])[0].get('physicalLocation', {}).get('artifactLocation', {}).get('uri', '')
|
|
vulns.append(f'- {rule_id} in {loc}')
|
|
print(json.dumps('\n'.join(vulns[:20])))
|
|
except Exception:
|
|
print(json.dumps(''))
|
|
")
|
|
STATUS="[{\"source\":\"osv scan\",\"results\":[{\"kind\":\"warning\",\"title\":\"OSV vulnerability scan\",\"summary\":\"${VULN_COUNT} known vulnerabilit${VULN_PLURAL} found in pinned dependencies.\",\"detail\":${VULN_DETAIL},\"how_to_fix\":\"Review the findings in the [Security tab](../../security/code-scanning). Update the affected dependencies if a patched version is available.\"}]}]"
|
|
else
|
|
STATUS="[]"
|
|
fi
|
|
fi
|
|
|
|
echo "review_status=${STATUS}" >> "$GITHUB_OUTPUT"
|