No inspiration for prompts? No problem, outsource this task to a local llm.

I've made this comfyui workflow for fun and to test various nodes, it should be easy to setup and to run. Set the checkpoint, optional loras and the llm and run it. No prompting needed. It should produce surreal paintings like the samples.

Edit the instructions for the llm at will and HAVE FUN!

Mandatory Steps( details in the Red Notes):

1 - Set checkpoint

2 - Set random loras (or just clear the current lists)

3 - Set llm (links to download models I've had success with in the Notes)

Optional Steps(details in the Tan Notes):

4 - Set helper loras (they REALLY help)

5 - Adjust llm instruction/prompt

6 - Experiment with various samplers/schedulers/steps/cfg

7 - Drive generations with a ControlNet (VERY fun to do!)

8 - Drive generations with IPAdapter

9 - Generate a video (optional prompt scheduler) - increase NrOfFrames in multiple of 16 and add a VideoCombiner after the sampler

A - Simple Instructions for creating a prompt

B - 'Art prompting manual' as instruction for creating a prompt

^ These are NOT set in stone, I barely know what I'm doing. Play around and experiment with instructions/settings for the llm.

The workflow uses 15, yes FIFTEEN custom nodes, sorry for that, but testing nodes was one of my goals with this workflow. Most of them are REALLY useful anyway (like WAS, ComfyRoll, ImpactPack etc). I'm sure it could be at least halved, there are now multiple options for the same functionality.

If you can afford API calls, there are custom nodes you could replace the "local" ones with "api nodes".

V2: is meant as a showcase on how you can play around with the instructions for the llm. I've added two extra painters that are now trying to paint "absurd art" and "novel combos". Feel free to edit them and experiment with your own ideas.

Aaaaaaaand, don't forget to have fun!

V3: IPAdapter apr2024 update