mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-05-03 02:11:48 +00:00
fix(skills/comfyui): bug fixes, cloud parity, expanded coverage, examples, tests
The audit of v4.1 surfaced ~70 issues across the five scripts and three
reference docs — most user-visible (silent file overwrites, status-error
misclassified as success, X-API-Key leaked to S3 on /api/view redirect,
Cloud endpoints that 404 because they were renamed). v5.0.0 fixes those
and fills the gaps that previously forced users to write their own glue
(WebSocket monitoring, batch/sweep, img2img upload helper, dep auto-fix,
log fetch, health check, example workflows).
Critical fixes
- run_workflow.py: poll_status now checks status_str==error BEFORE
completed:true, so a failed run no longer reports success
- run_workflow.py: download_output streams to disk via safe_path_join,
preserves server subfolder structure (no silent overwrites), and
retries with exponential backoff
- run_workflow.py: refuses to overwrite a link with a literal in
inject_params (would silently break wiring)
- _common.py: _StripSensitiveOnRedirectSession (subclasses
requests.Session.rebuild_auth) drops X-API-Key/Cookie on cross-host
redirects — fixes a real key-leak path through Cloud's signed-URL
download flow. Tested
- Cloud routing (verified live): /history → /history_v2,
/models/<f> → /experiment/models/<f>, plus folder aliases for the
unet ↔ diffusion_models and clip ↔ text_encoders rename
- check_deps.py: distinguishes 200/empty vs 404 folder_not_found vs
403 free-tier; emits concrete fix_command per missing dep
- extract_schema.py: prompt vs negative_prompt determined by tracing
KSampler.{positive,negative} connections (incl. through Reroute /
Primitive nodes) instead of meta-title heuristic; symmetric
duplicate-name resolution; cycle-safe trace_to_node
- hardware_check.py: multi-GPU pick-best, Apple variant detection,
Rosetta detection, WSL2, ROCm --json, disk-space check, optional
PyTorch probe; powershell preferred over deprecated wmic
- comfyui_setup.sh: prefers pipx → uvx → pip --user (with PEP-668
fallback); idempotent — skips relaunch if server already up;
configurable port/workspace; persistent log; SIGINT trap
New scripts
- run_batch.py — count or sweep (cartesian product), parallel up to
cloud tier limit
- ws_monitor.py — real-time WebSocket viewer; saves preview frames
- auto_fix_deps.py — runs comfy node install / model download for
whatever check_deps reports missing (with --dry-run)
- health_check.py — single command that runs the verification checklist
(comfy-cli + server + checkpoints + optional smoke test that cancels
itself to avoid burning compute)
- fetch_logs.py — pull traceback / status messages for a prompt_id
Coverage expansion
- Param patterns now cover Flux (BasicScheduler, BasicGuider,
RandomNoise, ModelSamplingFlux), SD3, Wan/Hunyuan/LTX video,
IPAdapter, rgthree, easy-use, AnimateDiff
- Embedding refs in CLIPTextEncode strings extracted as model deps
- ckpt_name / vae_name / lora_name / unet_name now controllable so
workflows can be retargeted per run
Examples
- workflows/{sd15,sdxl,flux_dev}_txt2img.json
- workflows/sdxl_{img2img,inpaint}.json
- workflows/upscale_4x.json
- workflows/{animatediff_video,wan_video_t2v}.json + README
Tests
- 117 tests (105 unit + 8 cloud integration + 4 cross-host security)
- Cloud tests auto-skip without COMFY_CLOUD_API_KEY; verified end-to-end
against live cloud API
Backwards compatibility
- All existing CLI flags continue to work; new behavior is opt-in
(--ws, --input-image, --randomize-seed, --flat-output, etc.)
This commit is contained in:
parent
7d48a16f14
commit
a7780fe05f
32 changed files with 6117 additions and 1372 deletions
86
skills/creative/comfyui/workflows/README.md
Normal file
86
skills/creative/comfyui/workflows/README.md
Normal file
|
|
@ -0,0 +1,86 @@
|
|||
# Example Workflows
|
||||
|
||||
These are starter API-format workflows for the most common tasks. They're
|
||||
ready to run with `scripts/run_workflow.py` once you've installed (or have
|
||||
cloud access to) the listed models.
|
||||
|
||||
| File | Purpose | Required models | Min VRAM |
|
||||
|------|---------|-----------------|----------|
|
||||
| `sd15_txt2img.json` | SD 1.5 text-to-image (512×512) | SD1.5 checkpoint, e.g. `v1-5-pruned-emaonly.safetensors` | 4 GB |
|
||||
| `sdxl_txt2img.json` | SDXL text-to-image (1024×1024) | `sd_xl_base_1.0.safetensors` | 8 GB |
|
||||
| `flux_dev_txt2img.json` | Flux Dev text-to-image (1024×1024) | `flux1-dev.safetensors`, `t5xxl_fp16.safetensors`, `clip_l.safetensors`, `ae.safetensors` | 24 GB (or use `flux1-dev-fp8`) |
|
||||
| `sdxl_img2img.json` | SDXL image-to-image | SDXL checkpoint | 8 GB |
|
||||
| `sdxl_inpaint.json` | SDXL inpainting (image + mask) | SDXL checkpoint | 8 GB |
|
||||
| `upscale_4x.json` | Standalone 4× ESRGAN upscale | `4x-UltraSharp.pth` (or any upscaler) | 4 GB |
|
||||
| `animatediff_video.json` | AnimateDiff text-to-video (16 frames) | SD1.5 checkpoint, `mm_sd_v15_v2.ckpt` motion module | 8 GB |
|
||||
| `wan_video_t2v.json` | Wan 2.x text-to-video (~33 frames) | `wan2.2_t2v_1.3B_fp16.safetensors`, `umt5_xxl_fp16.safetensors`, `wan_2.1_vae.safetensors` | 24 GB |
|
||||
|
||||
## Quick start
|
||||
|
||||
```bash
|
||||
# Run a workflow with prompt injection
|
||||
python3 ../scripts/run_workflow.py \
|
||||
--workflow sdxl_txt2img.json \
|
||||
--args '{"prompt": "majestic eagle in flight", "seed": 12345, "steps": 35}' \
|
||||
--output-dir ./out
|
||||
|
||||
# Img2img: upload an input image first via the script's helper
|
||||
python3 ../scripts/run_workflow.py \
|
||||
--workflow sdxl_img2img.json \
|
||||
--input-image image=./photo.png \
|
||||
--args '{"prompt": "make it watercolor", "denoise": 0.6}' \
|
||||
--output-dir ./out
|
||||
|
||||
# Cloud (set API key once)
|
||||
export COMFY_CLOUD_API_KEY="comfyui-..."
|
||||
python3 ../scripts/run_workflow.py \
|
||||
--workflow flux_dev_txt2img.json \
|
||||
--args '{"prompt": "a fox in a misty forest"}' \
|
||||
--host https://cloud.comfy.org \
|
||||
--output-dir ./out
|
||||
|
||||
# What can I tweak in this workflow?
|
||||
python3 ../scripts/extract_schema.py sdxl_txt2img.json --summary-only
|
||||
|
||||
# Are all required models / nodes installed?
|
||||
python3 ../scripts/check_deps.py wan_video_t2v.json
|
||||
```
|
||||
|
||||
## Notes
|
||||
|
||||
- **Inpaint masks**: white pixels = "regenerate this region", black = preserve.
|
||||
ComfyUI's `LoadImageMask` reads the **red channel** by default; export your
|
||||
mask as a single-channel image or as a normal RGB where red==intensity.
|
||||
|
||||
- **Denoise strength** in img2img: `0.0` = output identical to input,
|
||||
`1.0` = ignore input entirely. Sweet spot is usually 0.4–0.7.
|
||||
|
||||
- **Flux Dev** needs ~24 GB VRAM in its base form. The `flux1-dev-fp8.safetensors`
|
||||
variant (already on Comfy Cloud) cuts that roughly in half.
|
||||
|
||||
- **Video workflows** can take many minutes. The skill auto-detects video
|
||||
output nodes and bumps the default timeout to 900s. Override with `--timeout 1800`.
|
||||
|
||||
- These JSON files are deliberately **API format** (top-level keys are node IDs
|
||||
with `class_type`), not editor format. To open them in ComfyUI's web UI for
|
||||
visual editing, use `Workflow → Load (API Format)` or `Workflow → Open` and
|
||||
follow the prompt.
|
||||
|
||||
## Cloud vs local model names
|
||||
|
||||
Comfy Cloud's preinstalled checkpoints sometimes have a `-fp16` suffix
|
||||
(`v1-5-pruned-emaonly-fp16.safetensors`) while the canonical local download
|
||||
keeps the original name (`v1-5-pruned-emaonly.safetensors`). The example
|
||||
workflows use the local-canonical names. When running on cloud, override with:
|
||||
|
||||
```bash
|
||||
python3 ../scripts/run_workflow.py \
|
||||
--workflow sd15_txt2img.json \
|
||||
--args '{"ckpt_name": "v1-5-pruned-emaonly-fp16.safetensors", "prompt": "..."}' \
|
||||
--host https://cloud.comfy.org
|
||||
```
|
||||
|
||||
The `ckpt_name`, `vae_name`, `lora_name`, `unet_name`, etc. are all exposed
|
||||
as controllable parameters by `extract_schema.py` — discover what's installed
|
||||
with `comfy model list` (local) or `curl /api/experiment/models/checkpoints`
|
||||
(cloud).
|
||||
Loading…
Add table
Add a link
Reference in a new issue