Correct, that's why I do not recommend text to video for the current version. Maybe it'll work better in a later checkpoint, otherwise, only use for image to video.

Training text to video loras for such varied concepts usually requires very large datasets and a lot of training, in most cases it's better to train the whole model instead of a lora, as that can capture more details. No promises if text to video is going to work with this lora in the future either, but for now image to video works pretty well.

You can change your prompt, I used the same prompt in most examples because prompt encoding is very slow for me since it's running on cpu. I just wrote a generic prompt that would work. But you can prompt for other things as well.

https://civitai.com/models/2086389/uncanny-photorealism-chroma

Specifically converted to nvfp4 but I haven't uploaded that model. The nvfp4 I made is hit or miss sometimes, the actual model is really good.

@happylittleteapot Pretty excited for zeta chroma (https://huggingface.co/lodestones/Zeta-Chroma) tbh, haven't really seen a lot of details about it aside from that it's based on z-image, I think the architecture is a bit different though since I'm unable to load it in comfy, it's still early in training anyway I think but I'm excited to see where that goes.

(CivitAI put my reply to a different comment here?)
Not yet, I think I'll need to make a good captioner before I can get to that point, 200 videos is pretty good for image to video, but I feel like I still need more for text to video to really work.

@Gradasho yeah most loras focus on a single subject, so that can be trained pretty easily. With this lora however there are a lot of concepts involved. The design of loras also makes them more optimal for a single subject.

With a multi purpose lora you need multiple videos of each concept, it will take a lot of training for the model to understand what you want. Technically finetuning would work better here, but that has significantly higher hardware requirements.

But both due to the small (relative to usual finetuning) dataset with imperfect captions and the fact that it's a lora is largely holding it back and making it hard to do t2v.

There are some tricks to train with the memory usage of a lora but adjusting the full model, but I've only seen them used in LLM trainer tools so far.

@N3RDGURL You could tip them buzz on their profile if you have any.

I have 48gb ram and use a 96 to 192gb max pagefile. I actually got a second nvme drive just so I could increase my pagefile without relying too much on my main drive (which already doesn't have much storage left)

I use the nvfp4 ltx 2 dev with the distil lora for 8 steps 1 cfg with euler a + beta

Also my gens are raw gens, no refiner/latent upscale pass. It's possible that latent upscale has issues.

@ylvlylvuyv374212 Oh yeah I've noticed, I've also seen some people say their latent upscale refined results had better lipsync quality, but I got the same quality lipsync in a single full res gen without upscale used.

During latent preprocessing I used the regular Gemma 3 12b qat 4 model that ltx recommends yeah. Also all the generations are with nvfp4 Gemma from my huggingface repo (also not abliterated). In the end, abliterated Gemma knows less than the original, and since ltx 2 ignores the censorship since it doesn't sample like an llm but instead takes all processed tokens, the info is still there, uncensored.

Alright, a future version could possibly support text to video, but for now I'll include i2v in the title for clarity

@ForeverNecessary737716 Yep it's pretty universal, the dataset was primarily furry, with a smaller portion of human, for i2v this won't matter, it's not going to turn a human character into a furry or anything like that. For t2v it still knows what humans look like (although it's not good for t2v in general in the current state, at least not for sex, on its own).

1. All the example videos are generated using dev fp4 with distill lora at 8 steps in a single pass (2 step latent upscaler often doesn't look good with this lora, and it's not really faster that way anyway)

2. The lora was trained on various frame rate videos, any fps over 25 was lowered. 25fps is a supported fps for ltx 2. 24, 16 and 50 are some other values which work well, but 25 is the fps used in the training config template gen for example. Because of mixed fps training it should be less restrictive in framerates, I haven't had it break at the fps options I named earlier. Training exclusively on a single framerate could lead the model to forget how to handle other framerates (although risk is lower with lora).

I'm currently working on some captioning models, if that goes well I can make a much larger dataset with higher quality captions, maybe text to video will work then... I'll pre-test my captioner to caption a dataset for a smaller image model, maybe flux 2 Klein. If the anatomy stays somewhat stable then using it to caption videos should also work.

I would if I could. I'm working on captioning models first, currently on the 5th experiment. Trying to teach a vision llm nsfw concepts works but I still have to find the right configuration and tricks to not bias it too much it have it forget stuff.

Additionally to a larger, more high quality dataset, t2v training will take a lot longer and a lot more resources, I'd like to make a t2v version some day but I think right now that's way out of my budget.

I've got various experiments I plan on doing though, should give me an insight on things I can do for faster training, what works and what doesn't work. But t2v for ltx 2 is low priority now because it doesn't seem like it's in my budget right now, when I managed decent t2v on smaller models I'll start scaling up.

Hmm, okay I believe I've seen people recommend using the distil lora at 0.6 strength before, and -0.4 starting at distilled should have a similar effect to that.

The vae issues themselves were only in the distil model initially on release, the dev model did have the release vae, but the distil model had the beta vae still. I never ran into that myself though since I used dev with the distil lora.

@kunde2 16gb, comfy's memory management makes it work. (I also have 48gb ram and 200gb swap just in case), I run the nvfp4 version btw. (Also I made an unofficial nvfp4 version of ltx2.3, which I'll make an updated lora for at some point, haven't checked if there's an official nvfp4 yet though)

I'm not sure about targetting voices, I didn't caption voices super specifically, you can try giving it a description but you'll have to rely on the base model's ability to understand what a voice should sound like from a description. Accents and stuff work pretty well though iirc.

Well if they're nsfw workflows, this lora basically handles nsfw motions and also tries to improve sound. Unlike the more common type of nsfw lora which adds a single thing, this lora tries to add general understanding of penetration, moans, and some other related concepts.

@girlswithafros For sfw stuff you can use the default workflows, like the comfy official ones, the ones made by ltx, or others from people on civitai

@girlswithafros then you'll probably want to use this lora lol

@crombobular Yeah but first I have some other stuff to do. I'm planning on/have started writing my own training software because I'm always getting issues, missing features or just poor implementations. When I get that in a functional state I'll release it and use it to train an updated lora. I've already got a plan for the training but you probably won't see it very soon since I still have a lot of work to do. I might release an experiment I did for flux 2 Klein 9b, where I got pretty nice results using muon, it's convinced me that it is possible to create a t2v capable lora with a low budget, I just need to use an optimizer like muon, it worked very well with flux, so I expect similar results with ltx 2.

The model is the original censored model. It's good for prompt encoding (it keeps the original meanings), but not enhancing (it'll refuse when used for text generation). You could either use a different model for the enhancing, like abliterated, but still use the original model for prompt encoding. Or, although I would not recommend it since there will be some knowledge loss, you could use the abliterated Gemma for both.

Personally I'd either go with no enhancement at all, or otherwise using a different model for the enhancing, but nvfp4 regular gemma for prompt encoding.

Text to video is not expected to work. But if it's got issues on image to video (like being worse than no lora) there's probably an issue with civitai's generator.

Also I am considering training a new version made for ltx 2.3. The current version does apply to ltx 2.3 since the transformer architecture is largely unchanged but features' meanings have shifted with the additional training, so I should probably retrain. (Not that civitai's generator supports 2.3 yet anyways.)

Yes, I'm currently still collecting data since I want to use a fresh dataset.

Make sure you're using regular Gemma and not an altered version such as abliterated, There might also be some words that the model misinterprets as speech, hopefully that won't happen in the next version for ltx 2.3.

It's over 3 days in training right now, I'll probably end training tomorrow. t2v is better than in the previous version, although still far from perfect. I'm having some issues generally with 2.3 but unrelated to the lora, if the release is later than expected it's probably that I'm still finding the right settings.