Preparing datasets for LoRAs can be a pain in the ass. Being lazy, I attempted to automate as much of the process as possible.
First, I have a workflow that extracts videos from a folder, and converts their fps to a target framerate of your choice (default 24 fps for Hunyuan). To do that, it takes the least common multiple "lcm" of the original and target fps (for example, for 30 and 24 it's 120), then it uses FILM to interpolate up to the lcm of the two numbers, and then it keeps only the frames it needs to go back down to the target fps. It's a bit overkill, but if you computer can handle it, it can save a bit of hassle.
I had to create a custom node for the math, here it is if the manager can't find it: https://github.com/EmilioPlumed/ComfyUI-Math.
Second, I have another workflow that captions the videos in a folder. I have it set up to use Joy Captioner alpha 2 to get a verbose description, and 2 wd14 taggers to get tags from different points. Each of the 3 captions have a slider to select from which portion of the video to extract the description or the tags. Then, it eliminates duplicate tags and puts the text together in a .txt file.
The workflows do require you to rename your files to be numbered consecutively.
Description
FAQ
Comments (24)
Can it work with Hunyuan 3D?
All of these workflows are intended to prepare datasets for LoRA or maybe model training https://civitai.com/articles/9798/training-a-lora-for-hunyuan-video-on-windows. They don't actually generate videos from inference.
Hello! I'm having trouble adding the path, it keeps saying it's not valid and adds a number at the end of the path which rises every time I run the workflow. I'm probably missing something obvious
The workflow is set up to go over files that are numbered consecutively. For example, "vid (1).mp4", "vid (2).mp4" and so on. This way you can process everything in your dataset in one go. If you prefer to go over each file one by one, you can rip out all of the string concatenation stuff at the beginning and just load the videos or images you want to process.
Got it, thanks!
I'm stuck on the path error, like the previous question. I've tried A:\vid\vid1.mkv and A:/vid/vid1.mkv and some others, changing .mp4 to .mkv, etc. no joy. Can you give an example of the syntax for one video in the path and an example of a directory of videos? I'm just not familiar with string syntax in python
Update: I've figured out the syntax to load the videos, and they seem to process, but the save node fails with an error. I've tried using the video input dir and putting .txt in the second string box, but it errors and seems to default to the ComfyUI\output dir. What is the correct syntax? Thanks
@noodles3211z_ ComfyUI is very particular when saving files. For some reason, it only allows saving on the same drive, and a lot of nodes are restricted to saving in a subfolder of either ComfyUI\input or ComfyUI\output. Set the save folder to something like "temp\captions\vid "+number+".txt" and when it's done copy the files to your dataset folder.
i downloaded your math nodes but i cant seem to get Exec Python Code Script working? its not loading in comfy, am i missing something?
I also installed python-interpreter-node as tensor_fanatic suggested
The python interpreter module should have done the trick. Try updating/restarting, and if it continues erroring out your best bet is debugging with chatGPT.
Thanks for letting me know it was that package, looking at the repository helped.
"To install via manager, you must first open the ComfyUI Manager settings and set the channel setting to dev:"
@neuraiai9377 Hi! I tried to switch channel to dev and restarted comfyui. Anyway i cant download this node with git url. Manager shows me message "this action not allowed with this security lvl". How to fix it?
@JosephSD I remember having to change a configuration file in ComfyUI, I'm sure you'll find the details if you google that error message.
I got an error with the python node, here my fix:
unique = [] for token in text1.split(","): stripped_token = token.strip() if stripped_token not in unique: unique.append(stripped_token) text1.to(", ".join(unique))
code block dont work in comments :/
Hello. For some reason, the "Join caption location", "Tag 1 location" and "Tag 2 location" nodes are not being loaded when updating via the Manager. Another point is that when loading the workflow, it says that the SimpleMathSlider+ node was not found. Can you help me?
All three are slider nodes from "ComfyUI_essentials". It shows up in the manager for me.
Do not use this... Not good
Can you be more specific? Maybe I can improve the workflows.
LEave it to the quest for better porn to pull out the math geek and scientists out of us.
Amen 🤣
It could be better to add https://github.com/IuvenisSapiens/ComfyUI_Qwen2-VL-Instruct instead of WD14 tagger
Looks pretty neat, specially the fact that it can caption videos directly. Is it better than JoyCaption? Maybe it could replace both of them.
@bonetrousers I would be great if you can make a new version with Qwen2. Your workflow is super-duper!
I uploaded your workflow to Gemini Pro 2.5 to review. This is what Gemini said:
You've found an excellent and very clever workflow. I've analyzed the JSON file and the description, and you are absolutely right—this is a powerful tool designed to automate the most tedious parts of preparing a video dataset for LoRA training.
Damn straight, nice work!
Hi, thanks for this workflow! However, there is an Error showing up with me saying "VHS_LoadVideoPath: Cannot allocate array memory". The Videos are located in the ComfyUI-folder (as subfolder), so is the output folder, so I think it should work. What am I missing??