<h1 id="ai-music-video-workflow-(comfyui)">🎬 AI Music Video Workflow (ComfyUI)</h1>Turn your favorite tracks into fully AI-generated cinematic music videos — automatically right inside ComfyUI - NO POST EDITING NEEDED. This workflow takes a reference image and an audio file, then generates a lip synced video that matches lyrics, mood, and scene dynamics and is 95% fully automated.For some reason most of the example videos are not showing up, guessing they are too long. You can find them all <a target="_blank" rel="ugc" href="https://youtube.com/playlist?list=PLQ0zxAQhttlZpolPMJTeQQjafa__MaD2v&amp;si=jv-gu7hMONsuMwvy">here</a>High Level walkthrough can be found <a target="_blank" rel="ugc" href="https://youtu.be/ggZATYEe-d4">here</a>Need help or have questions? <a target="_blank" rel="ugc" href="https://discord.gg/FJ9VvCDXw3">Please reach out through discord</a><hr /><h2 id="what-it-does">✨ What It Does</h2><ul><li>🎭 Keeps your reference image as the main performer across all scenes.</li><li>🎶 Splits audio into lyric-synced snippets for perfect timing.</li><li>🖋️ Uses a custom Prompt creator node that sends custom instructions to an LLM node to build cinematic prompts from the lyrics and your style choices.</li><li>🎥 Generates scene-by-scene visuals, then combines them into a seamless final video.</li></ul>The samples I provided were all created inside ComfyUI with NO post edits.On a 5090 it took around 2 hours for the full song.More examples can be found <a target="_blank" rel="ugc" href="https://youtube.com/playlist?list=PLQ0zxAQhttlZpolPMJTeQQjafa__MaD2v&amp;si=jv-gu7hMONsuMwvy">here </a>and more will be added as I make them.<hr /><h2 id="key-features">🔧 Key Features</h2><ul><li>Reference Image Control – Import your character photo (headshot recommended) it auto-removes the background, and resizes for clean framing.</li><li>Audio Handling – Automatic vocal/instrument separation, Whisper V3 transcription, advanced settings for lyric overlap, and fallback options.</li><li>Prompt Creator – Flexible scene builder with fields for style, theme, lighting, camera motion, outfits, and more to get a custom look</li><li>Auto Queueing – Handles multi-run videos seamlessly for long audio files.</li><li>Final Render Automation – Collects all video chunks, merges them, and saves your finished video as <code>FINAL_VIDEO.mp4</code>.</li><li>This workflow uses the Native Gemini LLM API node by default, which receives detailed instructions generated by the Prompt Creator node. You can swap Gemini out for another LLM if you prefer, but the instruction sets are fairly complex, and most local models struggle to follow them reliably. If you’d rather not use an LLM at all, you can manually enter prompts instead—just reach out on Discord for extra guidance and tips. For context, I’ve spent only $5 so far, which has powered 50+ videos, and I still have credit left—so it’s been very cost-effective.</li></ul><hr /><h2 id="quick-start">🚀 Quick Start</h2><ol><li>Upload reference image</li><li>Load your audio file</li><li>Set your folder name (e.g., the song title).</li><li>Fill in Prompt Creator fields (style, mood, shots, etc.).</li><li>Hit Run — everything else is automated.<ul><li>The workflow will auto-queue middle runs for long audio files.</li><li>For the final pass, it will tell you which groups to mute.</li><li>Simply follow the on-screen instructions, hit run again, and the workflow finishes the process automatically. (You do not have to wait for runs to finish. You just mute and hit run once more.)</li></ul></li></ol><hr /><h2 id="creative-workflow-tip">🎵 Creative Workflow Tip</h2>Just like real music videos, you don’t have to stick to one pass. You can run the same audio file multiple times with different reference images or styles — for example:<ul><li>One pass with the lead singer as the performer.</li><li>Another pass featuring a band member or supporting character.</li><li>Additional passes experimenting with different themes, outfits, or camera styles.</li></ul>Later, you can edit these separate video runs together, cutting between performances or blending visual moods — exactly how professional music videos are produced with multiple takes.<hr /><h2 id="required-custom-nodes">📦 Required Custom Nodes</h2>This workflow relies on a set of custom nodes I built specifically for this workflow. You’ll need to install them before running the workflow:👉 <a target="_new" rel="ugc" href="https://github.com/vrgamegirl19/comfyui-vrgamedevgirl/tree/main">ComfyUI-VRGameDevGirl Custom Nodes (GitHub)</a>They can also be installed via the manager.These nodes handle:<ul><li>Audio splitting, transcription, and auto-queueing</li><li>Smart folder management and metadata tracking.</li><li>Popup instructions for multi-run projects.</li><li>Scene sync and frame adjustments for HuMo compatibility.</li><li>Video combining and more.</li></ul><hr />👉 <a target="_blank" rel="ugc" href="https://discord.gg/FJ9VvCDXw3">Join the discord community for support, tips and tricks.</a><hr /><h2 id="in-summary">✅ In Summary</h2>This workflow is designed for creators, musicians, and visual storytellers who want to merge AI visuals with music. With automatic transcription, smart prompt handling, and seamless video assembly, you can focus on creative direction while the workflow handles the heavy lifting.

Files with SHA256: 1d0b664c7bd1e057c728eec5e62cbf3c48ef0bde2dbdd2c7a734c01be4198ee6