These are the best optional files and versions of the 384 LTX2.3 1.1 distilled Lora and JoyAI Echo DMD deltas that I've made. Uploading them so people can use them on Civit comfy cloud.
DMD (v1) - Echo Delta extraction that is reshaped to match the 384 lora's shape. It has different sampling sigmas for video, does not drop in to the normal 8 step sigmas. Echo's DMD is fully superior to LTX's because they designed it to respect input and conditioning, it has very low adjustment of input images and frame_0 even at full strength, exposing massive flaws in LTX's official distillation that I've been critical of for a while. Sampling and implementation is still ongoing and in testing. Audio layers are replaced with the 384 distilled audio components because JoyAI totally screwed audio up.
Distilled Rank 72 energy - Dynamic re-rank of most components of 384 lora with the stronger layers remaining at 144, creates more motion than a normal 105 or 72 re-rank. More applicable for t2v or intense i2v motions.
Distilled Rank 72 condsafe - The opposite of energy, upper layers are reduced even more for a very weak and mild effect that interrupts conditioning changes in i2v.
Description
FAQ
Comments (33)
wtf dose dmd do?!?!?
Alternate distillation of the model done by JoyAI but only implemented in their dud finetune and fake nodes until I sprung it. Their method is superior to the Lightricks one. Out of the box the major advantage is the upscale pass stability and much sharper motion and night/day i2v stock likeness consistency. Sampling it still needs to be dialed in it's prone to hallucinations and prompt wandering and some color fading on upscale and also is much more oriented to realistic figures instead of vector or anime.
@tenstrip can i assume by motion you mean physics over all or just human motion?
@Ragamuffin20 All that is tied to the model's self attention and tied a lot to seed and prompt and also model, in LTX's case the dataset is sub par and the training is sub par with no reinforcement and it is only the most basic type of feed forward which is like a dinosaur of a video model nowadays. They should be doing auto regressive + MoE but at least the next version has experts. Until that it's just basically refine prompt and seed until its good, but DMD definitely hints at a more complete and smooth i2v output when it works.
It cannot be overstated how useful this file is for LTX-2.3: You will get significantly better prompt adherance and identity stability using this. Use this in place of any Distill LoRA. Will work with any sigmas, but you can snag the DMD set if you want.
Oh mate, can't wait to take this baby for a spin. Likeness preservation is definitely LTX's weakest link
@CFITprodigy You need the Face ID LoRA for maximum effect. It's still far from perfect, but a big improvement over vanilla LTX-2.3.
@boobkake22 roger that. Thanks for the tip!
@boobkake22 follow-up question if you don't mind, just to make sure I'm doing this right - I'm using a two-pass workflow that has one "Distilled LoRA" Load LoRA node for each sampling pass. Is this where these LoRAs should go, or should I slot (one of) them in the Power LoRA Loader node, in parallel with other LoRA?
Huge thanks, mate!
@CFITprodigy Hard to visualize what you're saying exactly. You can send the same model path to both samplers, if that's what you're asking. You'd only need to double things up if you need to adjust the data per pass - which you may want to do, in which case, branch with whatever LoRA loader you prefer and make your adjustments. Does that capture it?
Hey Man, I just want to thank you for the work you put into all of this. As a new person to this community, you've really made this a ton of fun to get great results without a ton of headache. Please let me know if I can support you in any way, truly appreciate you buddy!
https://ko-fi.com/tenstrip No problem when I started on sd1.5 it was others doing it so I just give back. I'm gone for a week but I'll be on a Lora spree after that, training on Eros 1.4 with that kofi funding some cloud training.
@tenstrip Sweet, thx dude! Enjoy your week off
Is lora strength still 0.5 if I use it in the ComfyUI default workflow? Or should it be higher? I'm asking because I usually use rank96_energy distilled lora at 0.5 strength with 15 second videos. DMD seems to be quite different.
You can use full strength with this one. At least under the reference conditions. Your milage may vary.
DMD is full strength in this current version. However you can go down to 0.8 in some instances to get a different effect.
Do I apply this on top of Dev 1.1 or distilled 1.1?
DMD is a full strength alternative it can't be used on distilled, only on undistilled or merged into it.
@tenstrip Thanks!
@czuster373 there is no official dev 1.1 only distilled 1.1 or distill lora 1.1
Excellent work; it works perfectly. Many thanks for the work and for making it available.
This is based on 1.1? So does it have the horrible audio inhalation distortion bug?
If so, I'll pass.
That is from people continuing to not use the audio weight at full strength. You can't just run the distilled lora at half strength, you need to split the video off at lower strength and run audio at full strength, that was never a bug that is collective user error.
@tenstrip oh. I dont use comfy, I use wan2gp. I guess i might let the creator know. I think it runs the whole lora at full strength though
Hi, thanks for the great work!
I've been experimenting with a 12-sigma schedule 1.0, 0.99375, 0.9875, 0.98125, 0.9700, 0.9250, 0.8300, 0.7000, 0.5500, 0.4000, 0.2500, 0.1250, 0.0) and wanted to share some feedback.
When using just the DMD LoRA, the overall visual quality is fantastic, but I sometimes get blurred motion or deformed fingers.
On the other hand, when I use a standard Distill LoRA (specifically ltx-2.3-22b-distilled-lora-dynamic_fro09_avg_rank_105_bf16), the base quality is just okayish, but the motion handling is significantly better.
To get the best of both worlds, I tried combining them: your DMD LoRA at 0.4 strength + the Distill LoRA at 0.4 strength. This actually gave me the absolute best results regarding both quality and motion! However, I do occasionally get small square/grid artifacts in the background.
Do you think it would be possible to train or merge a LoRA that combines the methods and strengths of both approaches?
Besides that, really amazing job on this!
Thanks for sharing, what sigma schedule do you use for the 2nd pass?
@DustAndDreams
0.85, 0.7250, 0.4219, 0.0
I also have a better approach for now, I'm using DMD for First Pass and Distill for 2nd Pass, best quality for me and no artifacts. I still hope it's possible to merge both together.
Is it a joke? It's surrealistic... The author starts the description of his work by 'it's his best jgggrrr poyuydh tyrtrtytyr he has made'. Huuuuu nice. And everybody to comment if its the uyiuy version or tytyt'. I'm so happy to know it.... Well, what is a jgggrrr poyuydh with a rank 878787878 ? It seems not important. What is the interest of it ? Full mystery. It's been 3 years that I'm working with ComfyUI and sometimes I have the impression that the people who are developing things have ever thought to explain things simply before throwing technical, useless details.
The 3 "Best" versions of the many alternate distilled versions I've made, there's a lot more on huggingface. Also the method and explanation is in like 3 plain english sentences for each. I've had 3 years of people not actually reading anything and leaving stupid comments on my models.
I had been using the 72cond lora for a while, but the post still took me a while to understand. I'll provide my notes in parentheses:
These are the best optional files and versions of the 384 LTX2.3 1.1 distilled Lora
(OP's adaptation of the official LTX2.3 distilled lora, version 1.1, rank 384 - those are the rank 72 files here)
and JoyAI Echo DMD deltas
(JoyAI Echo is a merge/retraining someone else made that was supposed to have really good video but bad audio and was previously incompatible with the standard ComfyUI LTX nodes - OP's lora version is the big DMD file here)
...
DMD (v1) - Echo Delta extraction that is reshaped to match the 384 lora's shape.
(so unlike base JoyAI this works with ComfyUI LTX nodes)
It has different sampling sigmas for video, does not drop in to the normal 8 step sigmas.
(so you should google JoyAI Echo sigmas and use a node that lets you specify manual sigmas for best results)
...
Audio layers are replaced with the 384 distilled audio components because JoyAI totally screwed audio up.
(so it just uses the official distilled lora's audio instead)
Distilled Rank 72 energy - ... (this is a drop-in replacement for the official distilled lora, I found it to overbake the output a bit, as expected)
Distilled Rank 72 condsafe - ... (I had been using this before instead of the official distilled lora and it's great)
I have to say all of these work very well for me, using them on top of Dasiwa LTX2.3 V3 Golden Lace model - V4 has too strong baked-in loras IMO. The DMD one is probably the greatest thing.
DMD doesn't work well. not with 10eros
Not for Eros 1.2 unless you run it at lower strength since 1.2 has internal remnants from a broad older JoyAI extraction inside it and already contained some DMD deltas. Eros 1.4 is literally made around using it so that'd be an very misleading statement.
@tenstrip I just tested it with eros 1.4 and the results are garbage. The person looks like a horror movie victim with t2v. I tried full and half strength. Same result just one is worse
Details
Files
LTX2.3_DMD_reshaped_r256.safetensors
Mirrors
LTX2.3_DMD_reshaped_r256.safetensors
LTX2.3_DMD_reshaped_r256.safetensors
LTX2.3_DMD_reshaped_r256.safetensors
LTX2.3_DMD_reshaped_r256.safetensors
LTX2.3_DMD_reshaped_r256.safetensors
LTX2.3_DMD_reshaped_r256.safetensors
LTX2.3_DMD_reshaped_r256.safetensors
LTX2.3_DMD_reshaped_r256.safetensors
LTX2.3_DMD_reshaped_r256.safetensors