Object Removal LoRA of Flux Fill Dev v2
Update 20250517
Released Remove Anything v1.0 on runninghub.
https://www.runninghub.ai/ai-detail/1923389726278283265
I could say much better than Object Removal LoRA v2
and it is stronger than many closed source platform on object removal.
bmm 4300 is Remove Anything v1.0
Update 20250503
Released v2 for object removal
Using new training method, select training objective by timestep, 700
Trained on subset of RORD dataset
Dataset around 500
epoch 3
repeat 1
rank 16
lr 1e-4
The last image in demo example is experimental and it introduces artifacts. So, the 700 reg 0.5 is deprecated.
Update 20240320
Released v1 for object removal
About this version
Trained with larger dataset.
Merged with additional inpaint training to improve details texture.
Known Issue:
Due to the training dataset mostly is to remove human, when removing part of human,
the model would tends to remove the human part from the image.
It is better to use original fill to accomplish this type of removal.
Update 20250316
Upload a more trained file.
It seems longer training improve the generalization.
Update 20250314
Adding random selection training for beta.
It ranfom selects noised factual or noised ground true to train.
It is able to prevent the model degradation due to mis-aligned training objective.
Trainer:
https://github.com/lrzjason/T2ITrainer
Dataset:
lrzjason/ObjectRemovalFluxFill
You could download the lora from huggingface without waiting. This EA is only to support me for further more open source development
https://huggingface.co/lrzjason/ObjectRemovalFluxFill
Model Description
This is an Object Removal LoRA fine-tuned from Flux Fill Dev model.
The lora is designed to remove objects from specified masked areas, making it useful for image editing tasks where unwanted objects need to be erased seamlessly.
This lora is inspired by Object Drop. Object Drop achieved amazing result on removing objects and I want to try it with Flux fill model.
Due to the computing power limitation, this alpha version only trained on very small dataset.
If anyone interested in and want to sponsor the computing power, please contact me.
Intended Use
This model is intended for non-commercial use only, as per the FLUX.1 [dev] Non-Commercial License.
Limitations
Non-Commercial Use Only: This model is restricted to non-commercial applications. Any commercial use is prohibited under the FLUX.1 [dev] Non-Commercial License.
Large Masked Area: The model may struggle with large masked area.
How to Use
To use this model, you need to provide an image and a corresponding mask indicating the area where the object should be removed. The model will then generate an edited image with the object removed.
Using Flux Fill as base model and load the lora.
Training Data
The model was fine-tuned on a small dataset of images with corresponding masks, focusing on object removal tasks. The dataset includes various objects and scenes to ensure robust performance.
License
This model is licensed under the [FLUX.1 [dev] Non-Commercial License](https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md). You are free to use, modify, and distribute the model for non-commercial purposes. Commercial use is strictly prohibited.
Citation
If you use this model in your research or projects, please cite it as follows:
```
bibtex
@misc{object-removal-lora,
author = {lrzjason},
title = {Object Removal LoRA of Flux Fill Dev},
year = {2025}
}
```
Contact
Twitter: [@Lrzjason](https://twitter.com/Lrzjason)
Email: [email protected]
QQ Group: 866612947
Civitai: [xiaozhijason](https://civarchive.com/user/xiaozhijason)
Wechat: fkdeai
Sponsors me for more open source projects:
Buymeacoffee:
Wechat:
Disclaimer: This model is provided "as-is" without any warranties. The authors are not responsible for any misuse or damages arising from its use.
Description
FAQ
Comments (15)
What computing power do you need to implement the full version?
First, I need to build up the synthesis dataset following the object drop paper. Try and error on the implementation. Then try a test version on larger dataset. I estimate it requires one to two weeks on 4090.
@xiaozhijason Should I train locally or cloud?
@Kappa_Neuro If you want to do similar task, you need a traing script for flux fill model. I trained the alpha version locally using my modified script which mainly referenced from diffusers and kohya
@xiaozhijason No, I'm talking about where did you train?
Can you teach in the cloud using more powerful hardware?
@Kappa_Neuro I trained on my own pc and using my own code.
@xiaozhijason You asked in the description to help you somehow.
So I want to understand how this can be done.
@Kappa_Neuro I don't know what you want to know. The logic is basically identical to ObjectDrop which described in description. Prepare dataset. Run the training script. Review the result. Adjust any problem if exists. If you want to train it by yourself, you could refer to ObjectDrop. I wouldn't release the training code now because of many things.
@xiaozhijason 他好像是想赞助你实现完整版的训练，不知道你一般在哪个平台训练，本地肯定是算力不够的
@lianghan34171 我看了作者的数据集，看起来不太好。 我们需要创建一个高质量的数据集。
@lianghan34171 这并不重要，我已经开源了训练脚本，有兴趣可以自己练
@Kappa_Neuro It is a technical demo. At that time, I couldn't open source the training code. But now you could set up the repo in any cloud platform to train with you own data.
Hi, I'm trying to use your LoRA weights (trained for object removal) with the official Flux repository's cli_fill.py. However, I'm encountering an issue where the generated images are pure noise.
I noticed that your demonstration uses ComfyUI with a specific inpainting node. Could you please help me understand:
1. What are the key differences between your ComfyUI implementation and the official Flux cli_fill.py?
2. What modifications would be needed to make your LoRA work with the official cli_fill.py?
3. Are there any specific parameters or preprocessing steps that need to be adjusted?
My current setup:
- Using official Flux repo
- Added your LoRA path to configs
- Running with cli_fill.py
- Input: original image + mask
- Output: pure noise
Thank you for your help!
1. I wasn't tested about cli_fill.py. The lora is trained as peft format using diffusers library as backend. You might check the keys of the lora and see if it match with the key in cli_fill.py loaded model or not.
2. Comfyui does some convertion between lora weight but I am not sure would it affect like this or not.
3. The comfyui workflow does a scale image to 1 mega pixels for generation.
@xiaozhijason Thanks!
Details
Files
Available On (1 platform)
Same model published on other platforms. May have additional downloads or version variants.