NEW 2vXpSwA7: anytest-v4 | openpose-v2_1 || abovzv: segment || bdsqlsz: canny | depth | lineart-anime | mlsdv2 | normal | normal-dsine | openpose | recolor | segment | segmentv2 | sketch | softedge | t2i-color-shuffle | tile-anime-α | tile-anime-β | tile-real || BRIA AI: bg-gen | canny | colorgrid | depth | NEW fill | openpose | recolor || CVL-Heidelberg: canny | depth || destitech: inpaint | inpaintv2 || diffusers: canny small | mid | full | depth small | mid | full | zoe || EcomXL: inpaint | softedge || Eugeoter: NEW noobai canny | depth | lineart-anime | lineart-real | mangaline | normal | scribble-pidi | scribble-hed | softedge-hed | tile | sdxl anime-canny | vidit-depth || h94: ip-adapter | ipa-vith | ipa+ | ipa+face || Hetaneko: pony canny | cannyv2 | color | depth | replicate | replicatev2 || HighCWu: canny-v3 || huchenlei: PuLID || Kataragi: canny | flatline | flatline-lora | inpaint | NEW ipa | line2color | line2color-lora | lineart | lineart-lora | NEW noob-ipa | recolor | recolor-lora | NEW rough-coating | tori29-blur | xdog-sketch || kohya-ss: real blur | canny | depth | anime blur | blur-beta | canny | depth | openpose | openposev2 | replicate | replicatev2 | scribble || PromeAI: lineart || ShermanG lineart || Stability.ai: canny | depth | recolor | revision | sketch || SargeZT: depth-16b-zoe | depth-faid-vidit | depth-zeed | depth-zoe | softedge | t2i-adapters binary | canny | color | depth | segmentation | softedge || TencentARC: canny | depth-midas | depth-zoe | lineart | openpose | recolor | sketch || TheMistoAI: mistoline | mistoline-lora || thibaud: openpose | openpose-lora || ttplanet: tile-real | tile-realv2 || NEW windsingai: pose | tile | tile-10w || xinsir: canny | cannyv2 | depth | openpose | openpose-twins | scribble | scribble-anime | tile | union | union-promax
ControlNetXL (CNXL) - A collection of Controlnet models for SDXL
(13.01.2025 - First NoobAI controlnets uploaded by Eugeoter)
(12.01.2025 - First Illustrious controlnets uploaded: windsingai-pose & -tile)
This collection strives to create a convenient download location of all currently available Controlnet models for SDXL. Please do read the version info for model specific instructions and further resources. If you are the author of one of these models and don't want it to appear here, please contact me to sort this out. The naming scheme of the files follows lllyasviel's from here https://huggingface.co/lllyasviel/sd_control_collection/tree/main.
CAUTION: The variants of controlnet models are marked as checkpoints only to make it possible to upload them all under one version, otherwise the already huge list would be even bigger. If you use downloading helpers the correct target folders are extensions/sd-webui-controlnet/models for automatic1111 and models/controlnet for forge/comfyui.
Requirements for Automatic1111: at least version 1.5.0, better upgrade to the latest version of 1.6.0. + sd-webui-controlnet extension v0.400+. Bugs or weird behaviour might occur. If you encounter any irregularities you can join us on our discord and ask for support or get in contact with the developers of Automatic1111/Controlnet via github/huggingface.
A huge thanks to all the authors, devs and contributors including but not limited to: abovzv, bdsqlsz, BRIA AI, CVL-Heidelberg, destitech, the diffusers institution, Eugeoter, h94, Hetaneko, HighCWu, huchenlei, lllyasviel, kataragi, kohya-ss, Mikubill, PromeAI, SargeZT, ShermanG, Stability.ai, TencentARC, TheMistoAI, thibaud, ttplanet, windsingai and xinsir.
Description
PuLID is an ip-adapter alike method to restore facial identity. It uses both insightface embedding and CLIP embedding similar to what ip-adapter faceid plus model does. However, there is an extra process of masking out the face from background environment using facexlib before passing image to CLIP. PuLID also uses Eva CLIP instead of normal CLIP. In the attn overrides, PuLID also does something more than IPAdapter, as it zero pads the tensor and adds to ortho of hidden states.
PuLID uses both evaclip embedding and insightface embedding as proj module input. So in order to use the feature, you need to install https://github.com/huchenlei/sd-webui-controlnet-evaclip extension.
ComfyUI implementations:
https://github.com/cubiq/PuLID_ComfyUI provided by cubiq, native ComfyUI implementation
https://github.com/ZHO-ZHO-ZHO/ComfyUI-PuLID-ZHO provided by ZHO, diffusers-based implementation
Original repository: https://huggingface.co/huchenlei/ipadapter_pulid
FAQ
Comments (5)
Which Lineart, tile and openpose models have been tested in PonyXL?
Lineart from bdsqlsz gives an error(
no model should give you an error when used on a pony checkpoint, as pony is based on the sdxl architecture as well. might be more an issue of your workflow perhaps? are you on comfyui? make sure everything is up2date and you are using the correct nodes (https://github.com/kohya-ss/ControlNet-LLLite-ComfyUI)
@eurotaku oh. I'm using Stable Diffusion Forge.
@QwiziRAM hm, weird, can't remember any issues with the builtin controlnet module of forge. can i invite you to join our discord for a more indepth support, there are many very competent and helpful users there
@eurotaku Can I get an invite to the server, please? I need some help with models in general, especially inpainting ones.
Details
Files
controlnetxlCNXL_huchenleiPulid.safetensors
Mirrors
ipAdapterPulidSdxl_ipAdapterPulidSdxl.safetensors
controlnetxlCNXL_huchenleiPulid.safetensors
controlnetxlCNXL_huchenleiPulid.safetensors
ip-adapter_pulid_sdxl_fp16.safetensors
ip-adapter_pulid_sdxl_fp16.safetensors
ip-adapter_pulid_sdxl_fp16.safetensors
ip-adapter_pulid_sdxl.safetensors
ip-adapter_pulid_sdxl_fp16.safetensors
ip-adapter_pulid_sdxl_fp16.safetensors
ip-adapter_pulid_sdxl_fp16.safetensors
ipAdapterPulidSdxl_ipAdapterPulidSdxl.safetensors
ip-adapter_pulid_sdxl_fp16.safetensors
ip-adapter_pulid_sdxl_fp16.safetensors
ip-adapter_pulid_sdxl_fp16.safetensors
ip-adapter_pulid_sdxl_fp16.safetensors
ip-adapter_pulid_sdxl_fp16.safetensors
