mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-07-31 19:16:29 +00:00
ci: live-updating PR review comment with structured job statuses
Replace the static comment-pending + comment-results two-job pattern
with a live-updating comment system that polls the GitHub Actions API
every 15s, re-assembles the review comment from whatever results are
available, and upserts it via the <!-- hermes-ci-review-bot --> marker.
The comment updates in real time as each job finishes — no waiting for
the full pipeline.
Every CI job that wants to appear in the review comment emits a
review_status output — a JSON array of objects, each with a source
and a results array:
[
{
"source": "review-label-gate",
"results": [
{"kind": "action_required", "title": "...", "summary": "...",
"how_to_fix": "..."},
{"kind": "info", "title": "...", "summary": "..."}
]
},
{
"source": "ci timing",
"results": [
{"kind": "warning", "title": "CI timings", "summary": "...",
"detail": "...", "link": "..."}
]
}
]
One job can emit multiple results of different kinds. The source field
is used to exclude the corresponding job from the synthesized error
list (case-insensitive, hyphen-normalized matching against GitHub
Actions job display names).
| job | source | kind (on failure) | section |
|----------------------------|--------------------------|---------------------------|----------------------|
| review-labels | review label gate | action_required / info | Action required |
| lockfile-diff | lockfile-diff | action_required | Action required |
| ci-timings | ci timing | warning / info | Warnings |
| supply-chain scan | supply chain | error / (none) | Job failures |
| supply-chain dep-bounds | supply chain | action_required / (none) | Action required |
| osv-scanner | osv scan | warning / (none) | Warnings |
| uv-lockfile-check | uv.lock check | action_required / (none) | Action required |
| history-check | unrelated histories | action_required | Action required |
| contributor-check | contributor attribution | action_required | Action required |
Jobs that find nothing emit [] (empty array) — no noise info items.
A single comment-live job polls the GitHub Actions API every 15s,
classifies jobs into (completed, pending), assembles the comment, and
upserts it. Merges review_status outputs from all needs jobs via
toJSON(needs.*.outputs.review_status), and downloads the ci-timings
artifact when it becomes available. Shows commit SHA + message below
the header.
The assembler has ZERO job-specific knowledge. It just:
1. collect_from_statuses() — flattens all nested status objects into ReviewItems
2. collect_failed_jobs() — synthesizes errors for failed jobs with no declared status
3. _attach_job_urls() — fills in per-job log links for ALL items
4. render_comment() — groups by severity, renders with group headers
Each item shows links inline next to the title: View report (job-emitted
URL) and View job (auto-attached logs link). Each info item is its own
collapsible <details> block.
# ૮ >ﻌ< ა ci review
running on abc1234 — commit message first line
## ❌ Job failures
### {title} · [View job](url)
{summary}
## ⚠️ Action required
### {title} · [View job](url)
{summary}
**How to fix:**
{how_to_fix}
## ⚠️ Warnings
### {title} · [View report](url) · [View job](url)
{summary}
{detail}
<details><summary>{title}</summary>
{content}
</details>
Still running 3 jobs: ci-timings, docker
- test_assemble_review_comment.py (48 tests): collect_from_statuses,
collect_failed_jobs with exclude_sources, _attach_job_urls,
render_comment (group headers, inline links, commit info, per-item
details, pending footer), assemble integration
- test_live_comment.py (16 tests): classify_jobs pure function
- test_timings_report.py (10 tests): generate_review_status nested format
- test_lockfile_diff.py (6 tests)
- test_classify_changes.py (32 tests, pre-existing)
This commit is contained in:
parent
d7b36070ef
commit
b9f82ed39f
19 changed files with 2696 additions and 323 deletions
|
|
@ -15,11 +15,11 @@ Usage (from a checkout that still has the base ref available):
|
|||
--output diff.md [--repo-root .]
|
||||
|
||||
Reads every ``package-lock.json`` tracked at either ref (top-level and
|
||||
nested — the repo has several), diffs each, and writes a Markdown report
|
||||
to ``--output``. Exits 0 always; an empty report file means "no version
|
||||
changes" (the caller uses that to decide whether to post/update the PR
|
||||
comment). The report embeds ``COMMENT_MARKER`` so the workflow can find
|
||||
and update its own previous comment instead of stacking new ones.
|
||||
nested — the repo has several), diffs each, and writes a Markdown fragment
|
||||
to ``--output``. Exits 0 always; an empty output file means "no version
|
||||
changes" (the caller uses that to decide whether to include the section).
|
||||
The fragment is consumed by ``scripts/ci/assemble_review_comment.py``,
|
||||
which wraps it in a section with a header and action note.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
|
@ -29,9 +29,6 @@ import json
|
|||
import subprocess
|
||||
import sys
|
||||
|
||||
# Hidden marker used to locate the bot's previous comment for in-place update.
|
||||
COMMENT_MARKER = "<!-- hermes-lockfile-diff -->"
|
||||
|
||||
|
||||
def parse_lockfile(text: str) -> dict[str, str]:
|
||||
"""Reduce lockfile JSON to ``{install path: version}``.
|
||||
|
|
@ -79,21 +76,24 @@ def _display_name(path: str) -> str:
|
|||
|
||||
|
||||
def render_markdown(diffs: dict[str, dict[str, list]]) -> str:
|
||||
"""Render per-lockfile diffs as a Markdown PR comment body.
|
||||
"""Render per-lockfile diffs as a Markdown fragment.
|
||||
|
||||
``diffs`` maps lockfile repo-path → the output of :func:`diff_locks`.
|
||||
Lockfiles with no version changes are omitted. Returns ``""`` when
|
||||
nothing changed anywhere (caller skips commenting entirely).
|
||||
nothing changed anywhere (caller skips the section entirely).
|
||||
|
||||
The output is a fragment — per-lockfile ``####`` subsections with
|
||||
tables — not a standalone comment. The ``assemble_review_comment``
|
||||
script wraps this in a section with its own header and action note,
|
||||
so no top-level header or comment marker is emitted here.
|
||||
"""
|
||||
sections = []
|
||||
total = 0
|
||||
for lockfile, d in sorted(diffs.items()):
|
||||
added, removed, updated = d["added"], d["removed"], d["updated"]
|
||||
n = len(added) + len(removed) + len(updated)
|
||||
if n == 0:
|
||||
continue
|
||||
total += n
|
||||
lines = [f"### `{lockfile}`", ""]
|
||||
lines = [f"#### `{lockfile}`", ""]
|
||||
lines.append("| Package | Before | After |")
|
||||
lines.append("| --- | --- | --- |")
|
||||
for path, old, new in updated:
|
||||
|
|
@ -107,13 +107,7 @@ def render_markdown(diffs: dict[str, dict[str, list]]) -> str:
|
|||
if not sections:
|
||||
return ""
|
||||
|
||||
header = (
|
||||
f"{COMMENT_MARKER}\n"
|
||||
f"## ⚠️ `package-lock.json` changes ({total} package"
|
||||
f"{'s' if total != 1 else ''})\n\n"
|
||||
"This PR changes locked npm dependency versions."
|
||||
)
|
||||
return header + "\n" + "\n\n".join(sections) + "\n"
|
||||
return "\n\n".join(sections) + "\n"
|
||||
|
||||
|
||||
def _git_show(ref: str, path: str, repo_root: str) -> str | None:
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue