mirrors/hermes-agent
1
0
Fork 0
mirror of https://github.com/NousResearch/hermes-agent.git synced 2026-05-03 02:11:48 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 5

fix(skills/comfyui): bug fixes, cloud parity, expanded coverage, examples, tests

Browse source 
The audit of v4.1 surfaced ~70 issues across the five scripts and three
reference docs — most user-visible (silent file overwrites, status-error
misclassified as success, X-API-Key leaked to S3 on /api/view redirect,
Cloud endpoints that 404 because they were renamed). v5.0.0 fixes those
and fills the gaps that previously forced users to write their own glue
(WebSocket monitoring, batch/sweep, img2img upload helper, dep auto-fix,
log fetch, health check, example workflows).

Critical fixes
- run_workflow.py: poll_status now checks status_str==error BEFORE
  completed:true, so a failed run no longer reports success
- run_workflow.py: download_output streams to disk via safe_path_join,
  preserves server subfolder structure (no silent overwrites), and
  retries with exponential backoff
- run_workflow.py: refuses to overwrite a link with a literal in
  inject_params (would silently break wiring)
- _common.py: _StripSensitiveOnRedirectSession (subclasses
  requests.Session.rebuild_auth) drops X-API-Key/Cookie on cross-host
  redirects — fixes a real key-leak path through Cloud's signed-URL
  download flow. Tested
- Cloud routing (verified live): /history → /history_v2,
  /models/<f> → /experiment/models/<f>, plus folder aliases for the
  unet ↔ diffusion_models and clip ↔ text_encoders rename
- check_deps.py: distinguishes 200/empty vs 404 folder_not_found vs
  403 free-tier; emits concrete fix_command per missing dep
- extract_schema.py: prompt vs negative_prompt determined by tracing
  KSampler.{positive,negative} connections (incl. through Reroute /
  Primitive nodes) instead of meta-title heuristic; symmetric
  duplicate-name resolution; cycle-safe trace_to_node
- hardware_check.py: multi-GPU pick-best, Apple variant detection,
  Rosetta detection, WSL2, ROCm --json, disk-space check, optional
  PyTorch probe; powershell preferred over deprecated wmic
- comfyui_setup.sh: prefers pipx → uvx → pip --user (with PEP-668
  fallback); idempotent — skips relaunch if server already up;
  configurable port/workspace; persistent log; SIGINT trap

New scripts
- run_batch.py — count or sweep (cartesian product), parallel up to
  cloud tier limit
- ws_monitor.py — real-time WebSocket viewer; saves preview frames
- auto_fix_deps.py — runs comfy node install / model download for
  whatever check_deps reports missing (with --dry-run)
- health_check.py — single command that runs the verification checklist
  (comfy-cli + server + checkpoints + optional smoke test that cancels
  itself to avoid burning compute)
- fetch_logs.py — pull traceback / status messages for a prompt_id

Coverage expansion
- Param patterns now cover Flux (BasicScheduler, BasicGuider,
  RandomNoise, ModelSamplingFlux), SD3, Wan/Hunyuan/LTX video,
  IPAdapter, rgthree, easy-use, AnimateDiff
- Embedding refs in CLIPTextEncode strings extracted as model deps
- ckpt_name / vae_name / lora_name / unet_name now controllable so
  workflows can be retargeted per run

Examples
- workflows/{sd15,sdxl,flux_dev}_txt2img.json
- workflows/sdxl_{img2img,inpaint}.json
- workflows/upscale_4x.json
- workflows/{animatediff_video,wan_video_t2v}.json + README

Tests
- 117 tests (105 unit + 8 cloud integration + 4 cross-host security)
- Cloud tests auto-skip without COMFY_CLOUD_API_KEY; verified end-to-end
  against live cloud API

Backwards compatibility
- All existing CLI flags continue to work; new behavior is opt-in
  (--ws, --input-image, --randomize-seed, --flat-output, etc.)
This commit is contained in:
SHL0MS 2026-04-29 20:50:52 -04:00 • committed by Teknium
parent 7d48a16f14
commit a7780fe05f
32 changed files with 6117 additions and 1372 deletions
Download patch file Download diff file Expand all files Collapse all files

230
skills/creative/comfyui/references/rest-api.md
View file

@ -1,16 +1,50 @@
# ComfyUI REST API Reference
# ComfyUI REST + WebSocket API Reference
ComfyUI exposes a REST API + WebSocket for workflow execution and management.
Same API surface for local servers and Comfy Cloud (with auth differences).
ComfyUI exposes a REST + WebSocket interface for workflow execution and
management. **The same surface is used locally and on Comfy Cloud, with
auth/path differences.**
## Connection
| | Local | Cloud |
| | Local ComfyUI | Comfy Cloud |
|---|---|---|
| Base URL | `http://127.0.0.1:8188` | `https://cloud.comfy.org` |
| Auth | None (or bearer token) | `X-API-Key` header |
| API path prefix | none (`/prompt`, `/view`, …) | `/api/...` (`/api/prompt`, `/api/view`, …) |
| Auth | none (or bearer token if configured) | `X-API-Key` header |
| WebSocket | `ws://host:port/ws?clientId={uuid}` | `wss://cloud.comfy.org/ws?clientId={uuid}&token={API_KEY}` |
| Output download | Direct bytes from `/view` | 302 redirect → signed URL (use `curl -L`) |
| `/api/view` response | direct bytes | 302 redirect → signed URL (use `curl -L`) |
The skill scripts route URLs automatically via `_common.resolve_url()`.
## Endpoint differences on Comfy Cloud
The cloud surface diverges from local ComfyUI in several ways. The skill
scripts handle these transparently; document them here so anyone calling
`curl` directly knows.
| Local path | Cloud path | Notes |
|------------|-----------|-------|
| `/system_stats` | `/api/system_stats` | Cloud version is **public** (no auth required) |
| `/object_info` | `/api/object_info` | **Paid tier only** — free returns 403 |
| `/queue` | `/api/queue` | Paid tier only |
| `/userdata` | `/api/userdata` | Paid tier only |
| `/prompt` (POST) | `/api/prompt` (POST) | Paid tier only |
| `/upload/image` | `/api/upload/image` | Paid tier only; `subfolder` accepted but ignored |
| `/upload/mask` | `/api/upload/mask` | Same as above |
| `/view` | `/api/view` | Paid tier only; **returns 302** to signed URL |
| `/history` | `/api/history_v2` | **Renamed**; old path returns 404 |
| `/history/{id}` | `/api/history_v2/{id}` or `/api/jobs/{id}` | Both work; `/jobs` returns full job |
| `/models` | `/api/experiment/models` | **Renamed** |
| `/models/{folder}` | `/api/experiment/models/{folder}` | **Renamed**; response shape differs (see below) |
### Cloud model-list response shape
- **Local:** `["a.safetensors", "b.safetensors", …]` — flat list of strings.
- **Cloud:** `[{"name": "a.safetensors", "pathIndex": 0}, …]` — list of objects.
- **Cloud 404 with `code: "folder_not_found"`** — folder is empty or unknown,
not an "endpoint missing" error. Distinguish by reading the body.
The skill helper `_common.parse_model_list()` normalizes both.
## Workflow Execution
@ -34,7 +68,8 @@ curl -X POST "https://cloud.comfy.org/api/prompt" \
{"prompt_id": "abc-123-def", "number": 1, "node_errors": {}}
```
If `node_errors` is non-empty, the workflow has validation errors (missing nodes, bad inputs).
If `node_errors` is non-empty, the workflow has validation errors (missing
nodes, bad inputs).
### Check Job Status (Cloud)
@ -43,133 +78,146 @@ curl -X GET "https://cloud.comfy.org/api/job/{prompt_id}/status" \
-H "X-API-Key: $COMFY_CLOUD_API_KEY"
```
| Status | Description |
|--------|-------------|
| `pending` | Queued, waiting to start |
| `in_progress` | Currently executing |
| `completed` | Finished successfully |
| `failed` | Encountered an error |
| `cancelled` | Cancelled by user |
| Status | Description |
| ------------- | ---------------------------------- |
| `pending` | Job is queued and waiting to start |
| `in_progress` | Job is currently executing |
| `completed` | Job finished successfully |
| `failed` | Job encountered an error |
| `cancelled` | Job was cancelled by user |
### Job detail with outputs (Cloud)
```bash
curl -X GET "https://cloud.comfy.org/api/jobs/{prompt_id}" \
-H "X-API-Key: $COMFY_CLOUD_API_KEY"
```
Response includes `outputs` keyed by node ID. Cloud uses `video` (singular)
in the output structure; local uses `videos` (plural). The skill scripts
accept both.
### Get History (Local)
```bash
# All history
curl -s "http://127.0.0.1:8188/history"
# Specific prompt
curl -s "http://127.0.0.1:8188/history/{prompt_id}"
curl -s "http://127.0.0.1:8188/history" # all
curl -s "http://127.0.0.1:8188/history/{id}" # one prompt_id
```
Response contains `outputs` keyed by node ID with file references.
Local entry shape:
```json
{
"<prompt_id>": {
"prompt": [...],
"outputs": {"<node_id>": {"images": [...]}},
"status": {
"status_str": "success" | "error",
"completed": true | false,
"messages": [["execution_start", {...}], ["execution_error", {...}], …]
}
}
}
```
**Important:** when reading status, check `status_str == "error"` BEFORE
checking `completed`, because both can be true for failed runs.
### Download Output
```bash
# Local
# Local (direct bytes)
curl -s "http://127.0.0.1:8188/view?filename=ComfyUI_00001_.png&subfolder=&type=output" \
-o output.png
# Cloud (follow redirect)
curl -L "https://cloud.comfy.org/api/view?filename=ComfyUI_00001_.png&subfolder=&type=output" \
# Cloud (302 → signed URL; -L follows; STRIP X-API-Key for the second hop)
curl -L "https://cloud.comfy.org/api/view?filename=...&type=output" \
-H "X-API-Key: $COMFY_CLOUD_API_KEY" \
-o output.png
```
---
The skill's `run_workflow.py` strips `X-API-Key` automatically on the
cross-host redirect, so the signed URL never sees your auth.
## WebSocket Monitoring
Connect to WebSocket for real-time execution progress.
### Connection
Connect for real-time execution events.
```bash
# Local
wscat -c "ws://127.0.0.1:8188/ws?clientId=MY-UUID"
# Cloud
wscat -c "wss://cloud.comfy.org/ws?clientId=MY-UUID&token=API_KEY"
wscat -c "wss://cloud.comfy.org/ws?clientId=MY-UUID&token=$COMFY_CLOUD_API_KEY"
```
### Message Types (JSON)
**Note:** on Cloud the `clientId` is currently ignored — all messages for a
user are broadcast to every connection. Filter messages client-side by
`data.prompt_id`.
### JSON Message Types
| Type | When | Key Fields |
|------|------|------------|
| `status` | Queue change | `queue_remaining` |
| `status` | Queue change | `status.exec_info.queue_remaining` |
| `notification` | User-friendly status string | `value` |
| `execution_start` | Workflow begins | `prompt_id` |
| `executing` | Node running | `node` (ID), `prompt_id` |
| `executing` | Node running (or end-of-run if `node` is null on local) | `node`, `prompt_id` |
| `progress` | Sampling steps | `node`, `value`, `max` |
| `executed` | Node output ready | `node`, `output` |
| `execution_cached` | Nodes skipped | `nodes` (list of IDs) |
| `progress_state` | Extended progress with per-node metadata | `nodes` (dict) |
| `executed` | Node output ready | `node`, `output` (with `images`/`video`/etc.) |
| `execution_cached` | Nodes skipped because of cache | `nodes` (list of IDs) |
| `execution_success` | All done | `prompt_id` |
| `execution_error` | Failure | `exception_type`, `exception_message`, `traceback` |
| `execution_error` | Failure | `exception_type`, `exception_message`, `traceback`, `node_id` |
| `execution_interrupted` | Cancelled | `prompt_id` |
When `executing` has `node: null`, the workflow is complete.
### Binary Frames (Preview Images)
### Binary Messages (Preview Images)
| Type code | Meaning |
|-----------|---------|
| `0x00000001` | `PREVIEW_IMAGE` — `[type:4][image_type:4][data]` (image_type 1=JPEG, 2=PNG) |
| `0x00000003` | `TEXT` — `[type:4][nid_len:4][nid][text]` (UTF-8) |
| `0x00000004` | `PREVIEW_IMAGE_WITH_METADATA` — `[type:4][meta_len:4][json][image_data]` |
Format: `[4B type][4B image_type: 1=JPEG, 2=PNG][image_data...]`
---
`scripts/ws_monitor.py --previews <dir>` saves preview frames to disk.
## File Upload
### Upload Image
```bash
# Image
curl -X POST "http://127.0.0.1:8188/upload/image" \
-F "image=@photo.png" \
-F "type=input" \
-F "overwrite=true"
```
-F "image=@photo.png" -F "type=input" -F "overwrite=true"
# Returns: {"name": "photo.png", "subfolder": "", "type": "input"}
Response: `{"name": "photo.png", "subfolder": "", "type": "input"}`
### Upload Mask
```bash
# Mask (linked to a previously uploaded image)
curl -X POST "http://127.0.0.1:8188/upload/mask" \
-F "image=@mask.png" \
-F "type=input" \
-F "image=@mask.png" -F "type=input" \
-F 'original_ref={"filename":"photo.png","subfolder":"","type":"input"}'
```
---
Cloud equivalent: prepend `https://cloud.comfy.org/api` and add `-H "X-API-Key: $COMFY_CLOUD_API_KEY"`.
## Node & Model Discovery
### Object Info (All Nodes)
```bash
# All node types and their input specs
curl -s "http://127.0.0.1:8188/object_info" | python3 -m json.tool
# Returns all node types with input/output definitions
# Specific node
curl -s "http://127.0.0.1:8188/object_info/KSampler"
# Returns info for one specific node type
```
### Models by Folder
```bash
# Models per folder (local)
curl -s "http://127.0.0.1:8188/models/checkpoints"
curl -s "http://127.0.0.1:8188/models/loras"
curl -s "http://127.0.0.1:8188/models/vae"
curl -s "http://127.0.0.1:8188/models/controlnet"
curl -s "http://127.0.0.1:8188/models/clip"
curl -s "http://127.0.0.1:8188/models/upscale_models"
curl -s "http://127.0.0.1:8188/models/embeddings"
# Models per folder (cloud — note the experimental prefix)
curl -s "https://cloud.comfy.org/api/experiment/models/checkpoints" \
-H "X-API-Key: $COMFY_CLOUD_API_KEY"
```
Returns arrays of filenames (relative to model folder).
---
## Queue Management
```bash
# View queue (running + pending)
# View queue
curl -s "http://127.0.0.1:8188/queue"
# Clear all pending
@ -177,21 +225,19 @@ curl -X POST "http://127.0.0.1:8188/queue" \
-H "Content-Type: application/json" \
-d '{"clear": true}'
# Delete specific items from queue
# Delete specific items
curl -X POST "http://127.0.0.1:8188/queue" \
-H "Content-Type: application/json" \
-d '{"delete": ["prompt_id_1", "prompt_id_2"]}'
# Cancel currently running job
# Cancel currently-running job
curl -X POST "http://127.0.0.1:8188/interrupt"
```
---
## System Management
```bash
# System stats (VRAM, RAM, GPU, versions)
# Stats (VRAM, RAM, GPU, ComfyUI version)
curl -s "http://127.0.0.1:8188/system_stats"
# Free GPU memory
@ -200,14 +246,13 @@ curl -X POST "http://127.0.0.1:8188/free" \
-d '{"unload_models": true, "free_memory": true}'
```
---
## ComfyUI-Manager Endpoints (Optional)
## ComfyUI Manager Endpoints (Optional)
These require ComfyUI-Manager installed.
These require ComfyUI-Manager installed. Useful for installing nodes/models
via the API instead of `comfy-cli`.
```bash
# Install custom node from git repo
# Install a custom node from a git URL
curl -X POST "http://127.0.0.1:8188/manager/queue/install" \
-H "Content-Type: application/json" \
-d '{"git_url": "https://github.com/user/comfyui-node.git"}'
@ -221,8 +266,6 @@ curl -X POST "http://127.0.0.1:8188/manager/queue/install_model" \
-d '{"url": "https://...", "path": "models/checkpoints", "filename": "model.safetensors"}'
```
---
## POST /prompt Payload Format
```json
@ -246,11 +289,24 @@ curl -X POST "http://127.0.0.1:8188/manager/queue/install_model" \
},
"client_id": "unique-uuid-for-ws-filtering",
"extra_data": {
"api_key_comfy_org": "optional-partner-node-key"
"api_key_comfy_org": "optional-PARTNER-NODE-key (NOT the cloud auth key)"
}
}
```
- `prompt`: The workflow graph (API format)
- `client_id`: UUID for WebSocket event filtering
- `extra_data.api_key_comfy_org`: Required for paid partner nodes (Flux Pro, Ideogram, etc.)
- `prompt`: workflow graph in API format
- `client_id`: UUID — local server uses it to filter WebSocket events; cloud
ignores it.
- `extra_data.api_key_comfy_org`: ONLY required when the workflow uses
partner nodes (Flux Pro, Ideogram, etc.). Don't conflate with `X-API-Key`.
## Error Categories (cloud `execution_error` `exception_type`)
| Type | Meaning |
|------|---------|
| `ValidationError` | Bad workflow / inputs (often nicer to surface from `node_errors`) |
| `ModelDownloadError` | Required model not available |
| `ImageDownloadError` | Failed to fetch input image from URL |
| `OOMError` | Out of GPU memory |
| `InsufficientFundsError` | Account balance too low (partner nodes) |
| `InactiveSubscriptionError` | Subscription not active |