Local LLMs + ComfyUI = Smart Prompt Engineering (Free & Private)
# Local LLMs + ComfyUI = Smart Prompt Engineering (Free & Private)
Version 1.2.3 is out — a major update with 210 styles, face segmentation, and full architecture support!
Tired of writing prompts by hand? Want a fully local, private loop that actually improves your results instead of just dumping more tags?
ComfyUI LLM Prompt Studio closes the full cycle:
Idea → Generate prompt → Render image → Critique with vision model → Auto-revise → Save only approved results with full metadata
Everything runs on your machine. No cloud, no API keys, no data leaving your PC.
---
## 🔥 What's New in 1.2.3
### 210 Style Presets (was 14!)
The Styles/ folder system replaces the single config file. Now you have 210 professionally crafted styles across 25 categories:
- Photography (25 presets): Amateur, Professional Studio, Photojournalism, Wedding, Travel, Sports/Action, Wildlife, Food, Concert, Film/Analog, Drone, Real Estate, Product, Vintage/Lomo, Night, Newborn/Family, and more — each with camera/lens EXIF guidance so prompts include realistic technical details (e.g. "85mm prime f/1.4" for portraits, "300mm f/2.8 at 1/1000s" for sports)
- Cinema (34 presets): 19 director styles (Kubrick, Nolan, Tarantino, Fincher, Villeneuve, Wong Kar-wai, Kurosawa, Tarkovsky, Coppola, Malick, Bong Joon-ho, Coen Brothers, PTA, Godard, Eisenstein, Lang...) + 12 film genres (Film Noir, Neo-Noir, Spaghetti Western, Slasher, Found Footage, Zombie, Documentary, Mockumentary, Movie Musical, Grindhouse, Arthouse)
- Art Movements, Asian Art, Traditional Media, Digital & Contemporary, Fantasy & Horror, Anime & Manga, Period & Style, Basic Styles — and more
Presets support inheritance extends up to 3 levels deep), so you can easily create variations.
### Face Detailer: Segmentation & Advanced Refinement
No more rectangular face masks! New detection modes give you precise control:
- *yolo_seg** — segmentation mask instead of bounding box for natural face contours
- *yolo_bbox_seg** — strong bbox detection + seg mask for the inpaint shape (best for 3/4-turn faces)
- *mask_shape** square / oval) for bbox modes
- *bbox_scale** (0.1–3.0) — expand/contract the crop to include jaw/neck
- *iterations** (1–10) — re-refine each face multiple times for stronger cleanup
- *match_color** luminance / lab / histogram) — blend the refined face back seamlessly (no more "face brighter than the frame" seams)
New YOLO detectors: YOLOv10/11/12 (n/s/m) and YOLO26 face weights included — all auto-registered, no config needed.
Gender filter: Restrict refinement to one gender with a lightweight classification model gender_filter = any / female / male).
### Full Architecture Support (not just SDXL!)
The pack now adapts to your checkpoint's base architecture:
- SD1.5 — proper conditioning, cfg 7.0 defaults
- SDXL / Pony / Illustrious — dual g/l conditioning, AYS SDXL for distilled families
- Flux / SD3 — best-effort conditioning + forced no-negative mode (CFG ~1)
Wire detected_architecture from Smart Loader into Writer/Scene Builder/Smart Multi-Clip for automatic adaptation.
### New Nodes
- Smart Parameters — recommends steps / cfg / sampler / scheduler from checkpoint family. Supports AYS SD1/SDXL/SVD + GITS schedulers natively. Auto-detects Lightning/Hyper step counts from filenames.
- KSampler (Hires Fix) — single node for base pass + optional hires upscale. Supports latent interpolation, latent models, and pixel-model paths. Fractional factors (1.25x, 1.5x) now work correctly.
- Smart Multi-Clip (was Multi-CLIP SDXL) — architecture-aware encoding for up to 4 prompt pairs.
### Writer Enhancements
- *use_preset_system_prompt** toggle — explicit control over whether a preset overrides your system prompt (no more surprises with customized prompts)
- *nsfw** toggle — safe-for-work by default, explicit content only when requested
- *prompt_format** natural / tags / weighted / structured / midjourney / booru) — control how the model phrases prompts
- *blend_styles** — mix up to 3 styles additively (e.g. "oil_painting, cyberpunk")
- *cv_scene_hint** — cheap OpenCV pre-analysis (scene type, lighting, face count) feeds context to the LLM so prompts fit the actual content
### LM Studio API Auto-Profile
Writer/Critic/Scene Builder now read the real model architecture from LM Studio's native API and apply per-architecture sampler overrides (e.g. Gemma → top_k=64). The strict/baseline split uses actual parameter counts instead of filename heuristics.
### VRAM Management
- *release_vram_after_run** (default on) — unloads the LLM model after each node so the diffusion pipeline gets VRAM back
- Global kill switch: LLM_PROMPT_STUDIO_KEEP_MODEL_LOADED=1 disables all releases
- Defensive release in sampler nodes catches models loaded manually in LM Studio UI
### Seed Control
control_after_generate (randomize / increment / decrement / fixed) now works on Writer seed, KSampler seed + hires_seed, and Face Detailer seed. Seeds actually reach LM Studio (fallback to OpenAI endpoint when native rejects it).
---
## 🧩 Main Nodes
- Writer — turns any idea (any language) into a proper prompt + optional face prompts for inpainting. 210 styles, orthogonal toggles, architecture adaptation.
- Image Critic — vision model scores the image against the prompt and returns concrete revision notes. Auto-feeds notes back to Writer via auto_loop.
- Smart Save — saves only approved images as JPEG with full metadata in EXIF (or without for privacy).
- Scene Builder — two-stage: describe an image → compose a new prompt from the description + your edits. Architecture-aware stage 2.
- Smart Loader — loads checkpoint, detects distillation family (DMD/LCM/Turbo/Hyper/Lightning/Flash/schnell/tcd/pcm) + base architecture (SDXL/SD1.5/Flux/SD3/Pony/Illustrious), can auto-apply LoRA (now including CLIP weights!).
- Library Loader + prompt library — save and reuse good scenes with atomic writes and deduplication.
- Smart Multi-Clip — architecture-aware encoding of up to 4 prompt pairs.
- Smart Parameters — recommends sampler params from checkpoint family.
- KSampler (Hires Fix) — base pass + hires upscale in one node.
- Face Detailer — detect faces, upscale, refine, composite back. Now with segmentation, gender filter, and color matching.
---
## ⚡ Special Features
- No-negative mode — auto-switches for distilled models (CFG ~1). Constraints rewritten as positive statements. Flux/SD3 force it automatically.
- Full LM Studio native v1 API + fallback to OpenAI-compatible endpoints. Handles gpu_offload / gpuOffload key differences across versions.
- Optional face prompts for FaceDetailer / inpainting. Style tags now apply to face prompts too.
- Debug logging (off by default, masks API keys and images).
- Orthogonal assembly — nsfw / prompt_format / negative / face / blend_styles are independent toggles, never stored per preset.
- Persistent YOLO cache — detection models load once per session, not on every pass.
- Tiled VAE decode in Face Detailer for large hires latents.
---
## 📋 Requirements
- ComfyUI (tested on 0.19.3 classic)
- LM Studio (or any local OpenAI-compatible server)
- Instruct model for writing prompts (≥7B recommended for structured output)
- Vision model for Critic and Scene Builder (Qwen2.5-VL, LLaVA, Gemma-3/4, etc.)
- Optional: ultralytics package for Face Detailer YOLO detection (falls back to Haar cascade if unavailable)
---
## 🚀 Installation
```bash
cd ComfyUI/custom_nodes
git clone https://github.com/Ivas1981/comfyui_llm_prompt_studio.git
```
Restart ComfyUI. You should see:
```
[LLMPromptStudio] Package loaded: Writer, Critic, Smart Save, Library Loader, Scene Builder, Smart Loader, Smart Multi-Clip, Smart Parameters, KSampler Hires Fix, Face Detailer.
```
---
## 🔗 Links
- GitHub: https://github.com/Ivas1981/comfyui_llm_prompt_studio
- Full changelog and documentation are in the repository
---
## 🎯 Recommended Workflows
Basic prompt loop:
```
Writer → (your generation) → Critic → Smart Save
(enable Auto-loop on Critic for automatic revision)
```
High-quality with hires:
```
Writer → Smart Loader → Smart Multi-Clip → KSampler (Hires Fix) → Face Detailer → Critic → Smart Save
```
From image to new prompt:
```
Scene Builder (stage 1 describe → stage 2 compose) → (your generation) → Critic → Smart Save
```
All work great with Smart Loader for automatic no-negative detection + architecture adaptation, and Smart Parameters for optimal sampler settings.
---
100% local. 100% private. No subscriptions.
If you try it — leave feedback, share workflows, or report issues on GitHub. Enjoy! 🚀
---
P.S. Huge thanks to everyone who tested and reported issues during the 1.0.0 → 1.2.3 evolution. The project went through 111 commits, 4 audit rounds, and countless bug fixes to get here. Special shoutout to the community for the segmentation and gender filter feature requests — they made Face Detailer significantly better!