This is a control net that can help you design beautiful landscape image;
and you should keep rule below, the rule show Object corresponding pixel value;
you should trans the color to the calss_id,and the controlnet condition image is a mask with class id;so you can use the color draw image but when you put it into controlnet you should trans it to class id mask;or you can directly draw condition image with class id but it might be hard because too dark;
And the condition mask should be 3 channels mask with each channel same.
"color_object_map": { "#87CEEB": { "en_name": "sky", "zh_name": "天空", "class_id": 0, "color_name": "SkyBlue" }, "#2E8B57": { "en_name": "tree", "zh_name": "树", "class_id": 1, "color_name": "SeaGreen4" }, "#00E5EE": { "en_name": "sea", "zh_name": "海洋", "class_id": 4, "color_name": "Turquoise2" }, "#006400": { "en_name": "mountain", "zh_name": "山", "class_id": 5, "color_name": "DarkGreen" }, "#B8860B": { "en_name": "desert", "zh_name": "沙漠", "class_id": 18, "color_name": "DarkGoldenrod" }, "#FF0000": { "en_name": "car", "zh_name": "汽车", "class_id": 29, "color_name": "Red1" }, "#963232": { "en_name": "beach", "zh_name": "沙滩", "class_id": 3, "color_name": "DarkRed1" }, "#449632": { "en_name": "grass", "zh_name": "草", "class_id": 2, "color_name": "DarkGreen" }, "#a5b2ab": { "en_name": "snow", "zh_name": "雪", "class_id": 22, "color_name": "gray" }, "#5a7628": { "en_name": "ground", "zh_name": "地面", "class_id": 12, "color_name": "brown" }, "#3a1c94": { "en_name": "lake", "zh_name": "湖", "class_id": 23, "color_name": "lakeblue" }, "#2934dc": { "en_name": "river", "zh_name": "河流", "class_id": 25, "color_name": "riverblue" }, "#cdcee2": { "en_name": "cloud", "zh_name": "云", "class_id": 35, "color_name": "cloudgray" } }
Description
This version is the base version
FAQ
Comments (20)
If you have any questions please leave a message here.
Can you explain more about how to make masks, maybe give a couple of screenshots of how to assign ColorId correctly, etc.
u can use the "class id " as the mask color like semantic segmentation. Or you could use a sample python code trans the color "87CEEB" to "0". If you like our work please mark stars for us , thank you
we will post some mask images later
@Gerchegg Hi, we upload the mask image.
so can I manually change "river" lets say to wood floor etc? If yes then how and in which folder it can be done?
@radimetov2000488 you can modify the mask directly.
is this also possible to use in fooocus?
@chisarimarco75677 Have not tried
could you provide a sample controlnet workflow applying this? Im new to such method, and don't know where to use these color id masks. Many thanks!
I can use a painter node to connect with KSampler as latent image input, as a way to draw and generate images corresponding to the painted plot. But how to use these color id masks with controlnet? Hope you can provide a sample image with the workflow. None of the images in the gallery here has workflows embedded so I don't know where to start xD. Thanks.
we will post some mask images later. If you like our works please mark some stars for us ,thank you!
@gw_coffeeai_cv sounds gr8!
@thanatos1111 Hi, we upload the mask image.
@gw_coffeeai_cv managed to generate some landscape photos with masks here. I simply load the checkpoint, the dark mask here and the controlnet model in webui, with some simple prompts. Results looks good! Not sure if I need extra steps to generate details in the photo though.
@thanatos1111 you could use config like https://civitai.com/models/252072/gwmbasemodelv1, and code a simple python to convert the color mask to the dark mask. Have fun with this model. And don't forget add some stars to us . Thank U
@gw_coffeeai_cv yup, the checkpoint I was using is your gwmbasemodel. Just posted my result there xD.
@thanatos1111 OK,have fun.
Is this general semantic segmentation for SD XL? I've only seen a 1.5 model for this and have been dying to try it with XL. It looks like you might've only implemented the landscape-related colors?
Do you have anything on github? If it's within my skillset, I'd offer to adapt some of the other semantic segments.
Also: ADE20K or COCO dataset?
we use our own dataset for training. This model is only work on the landscape. We may implement more classes later. And so sorry, we don't have open source code.
Details
Available On (1 platform)
Same model published on other platforms. May have additional downloads or version variants.



















