ci: live-updating PR review comment with structured job statuses

Replace the static comment-pending + comment-results two-job pattern
with a live-updating comment system that polls the GitHub Actions API
every 15s, re-assembles the review comment from whatever results are
available, and upserts it via the <!-- hermes-ci-review-bot --> marker.
The comment updates in real time as each job finishes — no waiting for
the full pipeline.

Every CI job that wants to appear in the review comment emits a
review_status output — a JSON array of objects, each with a source
and a results array:

    [
      {
        "source": "review-label-gate",
        "results": [
          {"kind": "action_required", "title": "...", "summary": "...",
           "how_to_fix": "..."},
          {"kind": "info", "title": "...", "summary": "..."}
        ]
      },
      {
        "source": "ci timing",
        "results": [
          {"kind": "warning", "title": "CI timings", "summary": "...",
           "detail": "...", "link": "..."}
        ]
      }
    ]

One job can emit multiple results of different kinds. The source field
is used to exclude the corresponding job from the synthesized error
list (case-insensitive, hyphen-normalized matching against GitHub
Actions job display names).

| job                        | source                   | kind (on failure)         | section              |
|----------------------------|--------------------------|---------------------------|----------------------|
| review-labels              | review label gate        | action_required / info    | Action required      |
| lockfile-diff              | lockfile-diff            | action_required           | Action required      |
| ci-timings                 | ci timing                | warning / info            | Warnings             |
| supply-chain scan          | supply chain             | error / (none)            | Job failures         |
| supply-chain dep-bounds    | supply chain             | action_required / (none)  | Action required      |
| osv-scanner                | osv scan                 | warning / (none)          | Warnings             |
| uv-lockfile-check          | uv.lock check            | action_required / (none)  | Action required      |
| history-check              | unrelated histories      | action_required           | Action required      |
| contributor-check          | contributor attribution  | action_required           | Action required      |

Jobs that find nothing emit [] (empty array) — no noise info items.

A single comment-live job polls the GitHub Actions API every 15s,
classifies jobs into (completed, pending), assembles the comment, and
upserts it. Merges review_status outputs from all needs jobs via
toJSON(needs.*.outputs.review_status), and downloads the ci-timings
artifact when it becomes available. Shows commit SHA + message below
the header.

The assembler has ZERO job-specific knowledge. It just:
1. collect_from_statuses() — flattens all nested status objects into ReviewItems
2. collect_failed_jobs() — synthesizes errors for failed jobs with no declared status
3. _attach_job_urls() — fills in per-job log links for ALL items
4. render_comment() — groups by severity, renders with group headers

Each item shows links inline next to the title: View report (job-emitted
URL) and View job (auto-attached logs link). Each info item is its own
collapsible <details> block.

    # ૮ >ﻌ< ა ci review

    running on abc1234 — commit message first line

    ## ❌ Job failures
    ### {title} · [View job](url)
    {summary}

    ## ⚠️ Action required
    ### {title} · [View job](url)
    {summary}
    **How to fix:**
    {how_to_fix}

    ## ⚠️ Warnings
    ### {title} · [View report](url) · [View job](url)
    {summary}
    {detail}

    <details><summary>{title}</summary>
    {content}
    </details>

    Still running 3 jobs: ci-timings, docker

- test_assemble_review_comment.py (48 tests): collect_from_statuses,
  collect_failed_jobs with exclude_sources, _attach_job_urls,
  render_comment (group headers, inline links, commit info, per-item
  details, pending footer), assemble integration
- test_live_comment.py (16 tests): classify_jobs pure function
- test_timings_report.py (10 tests): generate_review_status nested format
- test_lockfile_diff.py (6 tests)
- test_classify_changes.py (32 tests, pre-existing)
This commit is contained in:
ethernet 2026-07-16 18:17:49 -04:00
parent d7b36070ef
commit b9f82ed39f
19 changed files with 2696 additions and 323 deletions

View file

@ -15,11 +15,11 @@ Usage (from a checkout that still has the base ref available):
--output diff.md [--repo-root .]
Reads every ``package-lock.json`` tracked at either ref (top-level and
nested — the repo has several), diffs each, and writes a Markdown report
to ``--output``. Exits 0 always; an empty report file means "no version
changes" (the caller uses that to decide whether to post/update the PR
comment). The report embeds ``COMMENT_MARKER`` so the workflow can find
and update its own previous comment instead of stacking new ones.
nested — the repo has several), diffs each, and writes a Markdown fragment
to ``--output``. Exits 0 always; an empty output file means "no version
changes" (the caller uses that to decide whether to include the section).
The fragment is consumed by ``scripts/ci/assemble_review_comment.py``,
which wraps it in a section with a header and action note.
"""
from __future__ import annotations
@ -29,9 +29,6 @@ import json
import subprocess
import sys
# Hidden marker used to locate the bot's previous comment for in-place update.
COMMENT_MARKER = "<!-- hermes-lockfile-diff -->"
def parse_lockfile(text: str) -> dict[str, str]:
"""Reduce lockfile JSON to ``{install path: version}``.
@ -79,21 +76,24 @@ def _display_name(path: str) -> str:
def render_markdown(diffs: dict[str, dict[str, list]]) -> str:
"""Render per-lockfile diffs as a Markdown PR comment body.
"""Render per-lockfile diffs as a Markdown fragment.
``diffs`` maps lockfile repo-path → the output of :func:`diff_locks`.
Lockfiles with no version changes are omitted. Returns ``""`` when
nothing changed anywhere (caller skips commenting entirely).
nothing changed anywhere (caller skips the section entirely).
The output is a fragment — per-lockfile ``####`` subsections with
tables — not a standalone comment. The ``assemble_review_comment``
script wraps this in a section with its own header and action note,
so no top-level header or comment marker is emitted here.
"""
sections = []
total = 0
for lockfile, d in sorted(diffs.items()):
added, removed, updated = d["added"], d["removed"], d["updated"]
n = len(added) + len(removed) + len(updated)
if n == 0:
continue
total += n
lines = [f"### `{lockfile}`", ""]
lines = [f"#### `{lockfile}`", ""]
lines.append("| Package | Before | After |")
lines.append("| --- | --- | --- |")
for path, old, new in updated:
@ -107,13 +107,7 @@ def render_markdown(diffs: dict[str, dict[str, list]]) -> str:
if not sections:
return ""
header = (
f"{COMMENT_MARKER}\n"
f"## ⚠️ `package-lock.json` changes ({total} package"
f"{'s' if total != 1 else ''})\n\n"
"This PR changes locked npm dependency versions."
)
return header + "\n" + "\n\n".join(sections) + "\n"
return "\n\n".join(sections) + "\n"
def _git_show(ref: str, path: str, repo_root: str) -> str | None: