This is an experiment lora only train with 1 image.

adamw constant 1e-4

trained with my custom training script https://github.com/lrzjason/T2ITrainer

The lora is a dora with diffusers format.

You could load the lora with the custom hunyuanDit pipeline in T2ITrainer.

It added the SD3LoraLoaderMixin to the pipeline.

Code:

lora_state_dict = HunyuanDiTPipeline.lora_state_dict(lora_path) HunyuanDiTPipeline.load_lora_into_transformer(lora_state_dict,transformer_)