https://openart.ai/workflows/indras_mirror/lumina2sdxl-l-ollama-autoprompt--ipa/AnnzsbhXmkc7K0AQzbCv

This is a workflow that takes an uploaded image, automatically describes the image with Ollama (Llava 1.6 Works well) uses the Lumina ComfyUI wrapper and sends latent to SDXL lightning. Also uses IP Adapter to influence the final output. Can use up to 23gb VRAM but that can be fixed by getting rid of Ollama and just prompting normally. Also changing the SDXL-Lightning model to a model of choice, ie SD 1.5

https://github.com/kijai/ComfyUI-LuminaWrapper/tree/main

https://huggingface.co/Alpha-VLLM/Lumina-Next-SFT

https://ollama.com/library/llava

https://ollama.com/library/llava-phi3