Updating for Wan! All Wan models are compatible with 2.1 and 2.2.

It may be possible. For me not yet. I've tried doing the training with multiple caption styles, and there is a ton of style and likeness bleed over. Basically every woman generated looks like a mish-mash of all of them. I won't quit trying, but at this point I haven't had any luck.

@Suppressor Well then, I guess you got successful into mashing them all in 1! Surprisingly the same file size. Are the single ones higher quality output or similar?

@ItsThatTimeAgain It seems about the same to me. I was shocked, this thing trained for like a week, I expected it to be huge.

@Suppressor same here.I tried many caption styles but it won't work. amazing work!!! what's the trick to merge them into one.

@twoldogs I think it has to do with with dataset size and training time. I increased the size of the image set to 1.8gb of images, and 3.1gb of video clips. I ran them through Joy Caption 2, then let it train for 8 days.

I think the mistake I was making before was assuming I could train multiple concepts in the same amount of time as I trained single concepts. Lost track of the steps and epochs, but it was in the 6 digits on steps last time I remember looking.

@Suppressor I see. Do I need to use a separate folder for each concept, or should we use one large folder to store all the images and videos inside? Thank!!!

@twoldogs I dumped it all in one giant folder.

I forget to mention that I heavily captioned with trigger words, like "CGI Disney Princess Belle, Belle has long, wavy brown hair cascading over her shoulders, styled in a loose, elegant manner. Princess Belle's eyebrows are well-defined, and Disney Belle has a delicate, slightly upturned nose. CGI Belle is wearing a soft, golden-yellow dress with a V-neckline, which adds a warm, glowing effect to the image. CGI Princess Belle's lips are painted a soft pink, and Belle has a subtle, gentle smile.

I did string searches for her and she, and replaced them with the triggers for the character's name. Hope this helps.

@Suppressor thank a lot. I just used the name as trigger word;as the result I feel trigger word did almost nothing even after 50000 steps.

@Suppressor very good job, I'd say this is one of the best loras so far imo.

As for bleeding, I've been thinking about the concept of training loras with two or more characters together, for example if you want less bleeding you would make sure to train Cloud and Tifa in many scenes together interacting, not just alone. Maybe something worth trying if you haven't already.

@Redbird That's a really good point. From some movie screen grabs, I did have a tiny bit of training material doing just that, but I didn't focus on it. I am sure that would beef it up and eliminate some of the bleeding. The weirdest thing I have found is if more than one princess is in a scene, it works way better if they are different ethnicities. Don't know why... Same ethnicity? You have to prompt extensively for the differences between the two. For your idea to bear fruit, I think I would have to actually create and generate scenes of multiple princesses, since except for "Ralph Breaks the Internet" and some promo videos, there is very little overlap between them all. Maybe as people start to generate art from it, some of that could contribute to training material.

@Suppressor yeah exactly, I've noticed the same with they having to be very different ethnicity or in looks to help with the bleeding.

And yeah I was thinking it would be hard to collect images with more than one of these characters together

It would work better with FF7, Friends or something because all those characters are always together, so easier to find images with those. But that won't help much with new ideas of new characters being together for the first time haha.

Im trying something similar, i was training to make a single lora of the two i have published, but not merging, i wanted to train, and the results are not convincing me.. with x150 3sec clips , with num repeats 3 at step 4000 the loss starting to climb.. i think something is not working, do you have available the tensorBoard graph for the all in one princess training? I was asking gpt about It and It told me that combining drastically different scenes and povs can affect the training process, making the lora to allucinate when generating instead of generating something specific

@K3NK Sorry, I did not save the Tensorboard logs for this. I would if I knew it was helpful, what did you hope to gain from looking at it? I can only share my experience, and I strongly suspect Chat GPT has no idea how to do this well. Though I did not use radically different POV, I did make sure to get all sides images of every character in my LoRA. What I do know, is, I was at least at 100k steps before things started to shape up. Insane patience required.

@K3NK I was thinking further on your models, I checked them out and installed them on my system so I could check... I strongly suspect you are dealing with captioning issues. Hunyuan inherently understands a lot of POV from my experience, though it tends to screw up with "from behind" and "low angle shot." Did you use standardization images or videos in your training?

i'm using videos, and yes, i was captioning manually a bit wrong at first, the last ones are joycaption and now i'm training one i captioned 118 clips with LLaVA-Video-7B-Qwen2-captioner, although people complain that pov is hard to get, actually it wasn't for me.. pov wasn't working in your tests? i guess for action loras you need to keep the data set the more focused on a single action or maybe if i train for 8 days.. 😅

when was the first time you tested? i mean, i guess you tested after 24h of training, were the results bad? in this one i told you, the loss started to rise in the tensorboard so i cancelled it, also the test i did came up horrible, i guess i will stick to simpler single action loras... i wanted to create an all in one but the model is really amazing but maybe with better captions and a longer training? the truth is that i have seen worse results in the tests than with the ones i did with sideview and 69pov.

@K3NK So yeah, to directly address the training, here is exactly what happened. I thought I had something around 8th epoch. After I put it through rigorous prompting I realized it was shite and I couldn't release it. Around 12 epochs, I was like Hmmmm, this is a possibility, but I was getting ghosting, artifacts and lack of likeness. I let it lie for 4 more days, just training all day. Around the 56th epoch, it started being REALLY f**king solid. Like unbreakable even with LoRAs. I let it go for 2 more days, and finally after testing I decided it was baked enough. In all honesty, with some of the feedback I got, there is going to be a final train, I may let it just go for 2 weeks and see what I get.
IMO, It's not like Flux or SD3 where you can over bake it, it feels more like one of those models where the more you are willing to put into it, the more you will get.

@Suppressor how low did you get with the loss??
if you still have the oputput folder you could just check tensorboard, i really wanted to check the graph loss at the end of the training

@Suppressor That's so interested, as with the image based trains I've tried they just freeze after like 1500 steps, no movement just images at that point. Maybe I need to throw some clips into the mix or something

@Suppressor how big was your dataset? videos and pictures? im using like 58 clips of 3sec 24fs 256x256 and after 7 hours the loss stabilizes and doesnt go down.. im getting so sick of this... xD this one doesnt even have the mixture.. is just pov bj clips... but the graph looks almost the same as the others... were you using learning rate of 2?

@azeli Yeah, you will get freezing on static images after a point in my experience, gotta have some clips in there IMO.

@K3NK My dataset was 1.8gb of images, and 3.1gb of video clips.. Learning rate of 2, 100 warmup steps, video_clip_mode = 'single_middle'. I had many different sizes and quality of images and video.

Wow that's a lot of data! What rank was this? 32? Just to add some numbers on loss, it seems getting down to ~0.06-0.07 is good, and 0.05 seems overtrained.

@Suppressor if you Will train again please try to setup the tensorBoard,im very curious to see how the graph looks on such a Big dataset

@K3NK I'm trying to remember, is it just --log_with tensorboard on my train script to get it to write data to the directory?

@logenninefingers888 Yep, 32.

@Suppressor for me is joing to /output/folder and run the command:

tensorboard --logdir=.

@K3NK Hmm, I don't run locally, I am using a VSC tunnel to a training cluster. I think I can do it with the script, I will try this evening.