Tip appreciated if you enjoy the work, these take a lot of time to create! https://ko-fi.com/misticrain69
PhysLTX is a Lora that improves LTX's understanding of how objects should interact. Characters and objects show better impact, bounce and interaction physics. It reduces the weightless and floaty interactions objects and characters in LTX have with each other. If a character or object interacts with another it should noticeably be more grounded and have a sense of impact and friction. Trained on 2000 videos from the CLEVRER: CoLlision Events for Video REpresentation and Reasoning dataset
Yes this works the same for realistic and non NSFW generations as well
It responds well to normal prompts and prompts that describe impact and momentum. I use it at a strength of 1.0 but that can impact the audio quality for some reason, if you have audio quality issues lower the strength to around 0.7 or use a node to disable the audio influence from the Lora.
! If you post things in the gallery that violate the TOS I will block and report you. Don't do this shit.
For 18+ only. This is NOT to be used for any illegal or unethical purposes or on any real person or likeness. Don't be fucking sus.
Description
FAQ
Comments (21)
Thank you for your contributions to the LTX 2.3 ecosystem!
At strength 1.0 its blew out my audio (using dev transformer only mxfp8 block 32) but setting it to .70 solved the issue
thanks for your hard work.
It didn't work for me, even setting the weight to 0.5 didn't help; the sound was completely ruined. I tried turning off the Lora sound via the kj Lora node, but that didn't help either; I could only turn it off.
Try using the LTX2 Lora Loader Advanced node instead. it's kind of a pain, because I like the convenience of the multi-lora loaders for conserving canvas space, but using this node for each of your loras in a chain instead gives you control over how loras do (or don't) affect your audio.
This LTX lora loader allows you to reduce or eliminate the audio part of LORAs. https://github.com/Brojakhoeman/Loradaddyloaderltx
Strange I dont have this issue in wan2gp. Not sure why it would be doing that since audio training was disabled.
Ill try stripping the audio layers of the lora it should solve it.
Tried stripping the audio layers and while it does solve the audio issue it makes the lora weaker for some reason so I will keep it as is for now. Try adjusting strength or using the LTX lora loader node and reducing the audio part of the lora.
@MisticRain69 May I offer a suggestion? To you—or to other LoRA creators—I recommend against embedding audio into every single LoRA model. Instead, audio-specific LoRAs and action-specific LoRAs should be kept separate. Since LTX is already prone to glitches when stacking multiple LoRAs, having multiple active LoRAs that all contain audio results in a truly terrible user experience. Although it is currently possible to mitigate this issue by using nodes to lower the LoRA audio volume, the overall result remains quite poor. These are just my personal observations; I hope you will take them into consideration. Thank you.
@Silicon_Mirage At least in AI toolkit even with audio training disabled (Which I did for this lora) the audio layers remain and the audio layers also effect generated motions so simply stripping them out will fix the audio issue but very much weaken the Lora. So its not that simple currently since the audio layers are tangled with the video layers and things like AI toolkit dont allow you to simply never have any audio layers in the lora to begin with.
@Silicon_Mirage Muting works well via the LTX-2 Master Loader LoRa-Daddy
Yeah i'm having similar issues. At 0.4 i still hear some audio degradation, compared to without the lora
@MisticRain69 We can only hope that the developers will address and optimize this issue. Currently, the audio generation component is quite poor; even when using a node to reduce volume—which does offer some improvement—the resulting audio still contains a significant amount of background noise.
@NealPage Yes, this is indeed the node I am currently using. Although there has been some improvement, the audio still suffers from static and distortion.
@MisticRain69 I took a close look at the detailed specifications of various LoRAs today, and I noticed a pattern among those created by other authors:
If a LoRA is designed specifically for video content, its parameter value for audio influence is set to 0.
Conversely, if a LoRA is designed specifically for audio content, its parameter value for video influence is set to 0.
Given that so many users have reported significant audio distortion when using your LoRA, I checked its settings and discovered that your video parameter value is 1152, while your audio parameter value is 2112.
Although you mentioned in a previous reply that you had disabled audio training, I am still puzzled as to why the audio parameter value is higher than the video parameter value.
If you would like to use other authors' LoRAs as a reference for their video and audio parameter settings, you can find the "Loradaddyloaderltx" node—shared by another user in this comment section—to inspect them.
You can compare your LoRA against the following two examples:
The LoRA trained specifically for audio is named: hentai__voice_ltx23; its parameter value for video influence is 0.
The LoRA trained specifically for video is named: LTX2.3_Sott_Enhance; its parameter value for audio influence is 0.
@Silicon_Mirage Interesting because audio training was disabled from the beginning of training so the only thing I can think of that may be happening is a glitch in AI toolkit causing audio training to be enabled even with it set to disabled or EMA for some reason enabling audio even with audio training disabled. This was my AI toolkit training config. You will see num frames set at 1 but thats not true since it sets num frames in the config to 1 when auto frames is enabled even though its training on 81+frame videos. Do audio is disabled in the gui alongside all other audio settings.
--- job: "extension" config: name: "LTX2.3_physLTX" process: - type: "diffusion_trainer" training_folder: "C:\\AI-Toolkit\\output" sqlite_db_path: "./aitk_db.db" device: "cuda" trigger_word: null performance_log_every: 10 network: type: "lora" linear: 64 linear_alpha: 64 conv: 16 conv_alpha: 16 lokr_full_rank: true lokr_factor: -1 pretrained_lora_path: null network_kwargs: ignore_if_contains: [] save: dtype: "bf16" save_every: 100 max_step_saves_to_keep: 4444444 save_format: "diffusers" push_to_hub: false datasets: - folder_path: "C:\\AI-Toolkit\\datasets/physics" mask_path: null mask_min_value: 0.1 default_caption: "" caption_ext: "txt" caption_dropout_rate: 0 cache_latents_to_disk: true is_reg: false network_weight: 1 resolution: - 512 controls: [] shrink_video_to_frames: true num_frames: 1 flip_x: false flip_y: false num_repeats: 1 do_i2v: true fps: 24 auto_frame_count: true num_workers: 8 prefetch_factor: 8 pin_memory: true train: batch_size: 1 bypass_guidance_embedding: false steps: 4000 gradient_accumulation: 1 train_unet: true train_text_encoder: false gradient_checkpointing: true noise_scheduler: "flowmatch" optimizer: "adamw8bit" timestep_type: "weighted" content_or_style: "balanced" optimizer_params: weight_decay: 0.0001 unload_text_encoder: false cache_text_embeddings: true lr: 0.00007 ema_config: use_ema: true ema_decay: 0.999 skip_first_sample: false force_first_sample: false disable_sampling: true dtype: "bf16" diff_output_preservation: false diff_output_preservation_multiplier: 1 diff_output_preservation_class: "person" switch_boundary_every: 1 loss_type: "mse" audio_loss_multiplier: 1 logging: log_every: 1 use_ui_logger: true model: name_or_path: "Lightricks/LTX-2.3" compile: true quantize: true qtype: "qfloat8" quantize_te: true qtype_te: "uint4" arch: "ltx2.3" low_vram: true model_kwargs: {} layer_offloading: true layer_offloading_text_encoder_percent: 1 layer_offloading_transformer_percent: 0.9 attention_backend: "sdpa" sample: sampler: "flowmatch" sample_every: 323333333333 width: 768 height: 768 samples: [] neg: "" seed: 42 walk_seed: true guidance_scale: 4 sample_steps: 25 num_frames: 121 fps: 24 meta: name: "[name]" version: "1.0"@iodrg244 I'm a bit more bias towards this one that works really well:
https://github.com/phazei/ComfyUI-PowerLTXLoraLoaderExtra
Lets you easily sort and load many LoRAs at once.
@phazei This one is similar to that but newer. Might be more streamlined. https://github.com/808charlie/rgthree-comfy It's worked well for me so far.
@iodrg244 Well, I made ComfyUI-PowerLTXLoraLoaderExtra so I can definitely say that other one isn't more streamlined, and being a fork, makes it basically break the original RG3, which is just a entirely bad idea for the ecosystem. I did use RG3's power lora loader as guidance. Unlike so many of the nodes out there these days, I'm not a vibe coder and have been doing it for 30 years and made sure it was coded well.
@phazei I appreciate your work! Thanks for clarifying. I'll try yours :)
Recommended strength?
Details
Files
LTX2.3_Better_Physics_PhysLTX.safetensors
Mirrors
LTX2.3_Better_Physics_PhysLTX.safetensors
LTX2.3_Better_Physics_PhysLTX.safetensors
LTX2.3_Better_Physics_PhysLTX.safetensors
LTX2.3_Better_Physics_PhysLTX.safetensors
LTX2.3_Better_Physics_PhysLTX.safetensors
LTX2.3_Better_Physics_PhysLTX-mid_2592090-vid_2911845.safetensors
LTX2.3_Better_Physics_PhysLTX.safetensors
LTX2.3_Better_Physics_PhysLTX.safetensors