i wiill uplaod the compare img here after compleeedd
Description
๐ฟ Checking dataset... ๐MyDrive/Loras/sed/dataset ๐ Found 404 images with 2 repeats, equaling 808 steps. ๐ Divide 808 steps by 4 batch size to get 202.0 steps per epoch. ๐ฎ There will be 10 epochs, for around 2020 total training steps. ๐ญ Installing trainer... 59 packages can be upgraded. Run 'apt list --upgradable' to see them. W: Skipping acquire of configured file 'main/source/Sources' as repository 'https://r2u.stat.illinois.edu/ubuntu jammy InRelease' does not seem to provide it (sources.list entry misspelt?) The following additional packages will be installed: libaria2-0 libc-ares2 python3-pip-whl python3-setuptools-whl The following NEW packages will be installed: aria2 libaria2-0 libc-ares2 python3-pip-whl python3-setuptools-whl python3.10-venv 0 upgraded, 6 newly installed, 0 to remove and 59 not upgraded. Need to get 3,994 kB of archives. After this operation, 8,194 kB of additional disk space will be used. Selecting previously unselected package libc-ares2:amd64. (Reading database ... 122363 files and directories currently installed.) Preparing to unpack .../0-libc-ares2_1.18.1-1ubuntu0.22.04.3_amd64.deb ... Unpacking libc-ares2:amd64 (1.18.1-1ubuntu0.22.04.3) ... Selecting previously unselected package libaria2-0:amd64. Preparing to unpack .../1-libaria2-0_1.36.0-1_amd64.deb ... Unpacking libaria2-0:amd64 (1.36.0-1) ... Selecting previously unselected package aria2. Preparing to unpack .../2-aria2_1.36.0-1_amd64.deb ... Unpacking aria2 (1.36.0-1) ... Selecting previously unselected package python3-pip-whl. Preparing to unpack .../3-python3-pip-whl_22.0.2+dfsg-1ubuntu0.7_all.deb ... Unpacking python3-pip-whl (22.0.2+dfsg-1ubuntu0.7) ... Selecting previously unselected package python3-setuptools-whl. Preparing to unpack .../4-python3-setuptools-whl_68.1.2-2~jammy3_all.deb ... Unpacking python3-setuptools-whl (68.1.2-2~jammy3) ... Selecting previously unselected package python3.10-venv. Preparing to unpack .../5-python3.10-venv_3.10.12-1~22.04.15_amd64.deb ... Unpacking python3.10-venv (3.10.12-1~22.04.15) ... Setting up python3-setuptools-whl (68.1.2-2~jammy3) ... Setting up python3-pip-whl (22.0.2+dfsg-1ubuntu0.7) ... Setting up libc-ares2:amd64 (1.18.1-1ubuntu0.22.04.3) ... Setting up libaria2-0:amd64 (1.36.0-1) ... Setting up python3.10-venv (3.10.12-1~22.04.15) ... Setting up aria2 (1.36.0-1) ... Processing triggers for man-db (2.10.2-1) ... Processing triggers for libc-bin (2.35-0ubuntu3.8) ... /sbin/ldconfig.real: /usr/local/lib/libtbbbind.so.3 is not a symbolic link /sbin/ldconfig.real: /usr/local/lib/libur_adapter_level_zero_v2.so.0 is not a symbolic link /sbin/ldconfig.real: /usr/local/lib/libtbbmalloc.so.2 is not a symbolic link /sbin/ldconfig.real: /usr/local/lib/libtbbmalloc_proxy.so.2 is not a symbolic link /sbin/ldconfig.real: /usr/local/lib/libtcm_debug.so.1 is not a symbolic link /sbin/ldconfig.real: /usr/local/lib/libur_adapter_level_zero.so.0 is not a symbolic link /sbin/ldconfig.real: /usr/local/lib/libtcm.so.1 is not a symbolic link /sbin/ldconfig.real: /usr/local/lib/libumf.so.1 is not a symbolic link /sbin/ldconfig.real: /usr/local/lib/libur_adapter_opencl.so.0 is not a symbolic link /sbin/ldconfig.real: /usr/local/lib/libtbb.so.12 is not a symbolic link /sbin/ldconfig.real: /usr/local/lib/libtbbbind_2_5.so.3 is not a symbolic link /sbin/ldconfig.real: /usr/local/lib/libtbbbind_2_0.so.3 is not a symbolic link /sbin/ldconfig.real: /usr/local/lib/libur_loader.so.0 is not a symbolic link /sbin/ldconfig.real: /usr/local/lib/libhwloc.so.15 is not a symbolic link Cloning into '/content/trainer'... remote: Enumerating objects: 388, done. remote: Total 388 (delta 0), reused 0 (delta 0), pack-reused 388 (from 1) Receiving objects: 100% (388/388), 7.80 MiB | 17.90 MiB/s, done. Resolving deltas: 100% (223/223), done. HEAD is now at 47ef6ed added installer for linux python 3.12 added a log-like output of the warmup steps in the console Submodule 'sd_scripts' (https://github.com/kohya-ss/sd-scripts) registered for path 'sd_scripts' Cloning into '/content/trainer/sd_scripts'... Submodule path 'sd_scripts': checked out 'e89653975ddf429cdf0c0fd268da0a5a3e8dba1f' creating venv and installing requirements Looking in indexes: https://download.pytorch.org/whl/cu124 Collecting torch==2.5.1 WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x79ca0cf6ea40>: Failed to establish a new connection: [Errno 101] Network is unreachable')': /whl/cu124/torch-2.5.1%2Bcu124-cp310-cp310-linux_x86_64.whl Downloading https://download-r2.pytorch.org/whl/cu124/torch-2.5.1%2Bcu124-cp310-cp310-linux_x86_64.whl (908.3 MB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 908.3/908.3 MB 1.3 MB/s eta 0:00:00 Collecting torchvision==0.20.1 Downloading https://download-r2.pytorch.org/whl/cu124/torchvision-0.20.1%2Bcu124-cp310-cp310-linux_x86_64.whl (7.3 MB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 7.3/7.3 MB 96.4 MB/s eta 0:00:00 Collecting nvidia-nvjitlink-cu12==12.4.127 Downloading https://download.pytorch.org/whl/cu124/nvidia_nvjitlink_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl (21.1 MB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 21.1/21.1 MB 74.1 MB/s eta 0:00:00 Collecting nvidia-cusolver-cu12==11.6.1.9 Downloading https://download.pytorch.org/whl/cu124/nvidia_cusolver_cu12-11.6.1.9-py3-none-manylinux2014_x86_64.whl (127.9 MB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 127.9/127.9 MB 7.7 MB/s eta 0:00:00 Collecting nvidia-cudnn-cu12==9.1.0.70 Downloading https://download.pytorch.org/whl/cu124/nvidia_cudnn_cu12-9.1.0.70-py3-none-manylinux2014_x86_64.whl (664.8 MB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 664.8/664.8 MB 930.6 kB/s eta 0:00:00 Collecting nvidia-nvtx-cu12==12.4.127 Downloading https://download.pytorch.org/whl/cu124/nvidia_nvtx_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl (99 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 99.1/99.1 KB 15.0 MB/s eta 0:00:00 Collecting filelock Downloading filelock-3.29.0-py3-none-any.whl (39 kB) Collecting nvidia-cublas-cu12==12.4.5.8 Downloading https://download.pytorch.org/whl/cu124/nvidia_cublas_cu12-12.4.5.8-py3-none-manylinux2014_x86_64.whl (363.4 MB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 363.4/363.4 MB 4.1 MB/s eta 0:00:00 Collecting nvidia-cufft-cu12==11.2.1.3 Downloading https://download.pytorch.org/whl/cu124/nvidia_cufft_cu12-11.2.1.3-py3-none-manylinux2014_x86_64.whl (211.5 MB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 211.5/211.5 MB 5.6 MB/s eta 0:00:00 Collecting triton==3.1.0 Downloading https://download-r2.pytorch.org/whl/triton-3.1.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (209.5 MB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 209.5/209.5 MB 6.0 MB/s eta 0:00:00 Collecting networkx Downloading networkx-3.4.2-py3-none-any.whl (1.7 MB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 1.7/1.7 MB 8.0 MB/s eta 0:00:00 Collecting nvidia-cusparse-cu12==12.3.1.170 Downloading https://download.pytorch.org/whl/cu124/nvidia_cusparse_cu12-12.3.1.170-py3-none-manylinux2014_x86_64.whl (207.5 MB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 207.5/207.5 MB 6.5 MB/s eta 0:00:00 Collecting nvidia-curand-cu12==10.3.5.147 Downloading https://download.pytorch.org/whl/cu124/nvidia_curand_cu12-10.3.5.147-py3-none-manylinux2014_x86_64.whl (56.3 MB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 56.3/56.3 MB 11.9 MB/s eta 0:00:00 Collecting nvidia-nccl-cu12==2.21.5 Downloading https://download.pytorch.org/whl/nvidia_nccl_cu12-2.21.5-py3-none-manylinux2014_x86_64.whl (188.7 MB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 188.7/188.7 MB 2.1 MB/s eta 0:00:00 Collecting nvidia-cuda-nvrtc-cu12==12.4.127 Downloading https://download.pytorch.org/whl/cu124/nvidia_cuda_nvrtc_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl (24.6 MB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 24.6/24.6 MB 60.2 MB/s eta 0:00:00 Collecting sympy==1.13.1 Downloading sympy-1.13.1-py3-none-any.whl (6.2 MB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 6.2/6.2 MB 20.4 MB/s eta 0:00:00 Collecting fsspec Downloading fsspec-2026.4.0-py3-none-any.whl (203 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 203.4/203.4 KB 29.8 MB/s eta 0:00:00 Collecting jinja2 Downloading https://download.pytorch.org/whl/jinja2-3.1.6-py3-none-any.whl (134 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 134.9/134.9 KB 21.8 MB/s eta 0:00:00 Collecting nvidia-cuda-cupti-cu12==12.4.127 Downloading https://download.pytorch.org/whl/cu124/nvidia_cuda_cupti_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl (13.8 MB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 13.8/13.8 MB 97.3 MB/s eta 0:00:00 Collecting nvidia-cuda-runtime-cu12==12.4.127 Downloading https://download.pytorch.org/whl/cu124/nvidia_cuda_runtime_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl (883 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 883.7/883.7 KB 70.3 MB/s eta 0:00:00 Collecting typing-extensions>=4.8.0 Downloading https://download.pytorch.org/whl/typing_extensions-4.15.0-py3-none-any.whl (44 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 44.6/44.6 KB 7.3 MB/s eta 0:00:00 Collecting numpy Downloading numpy-2.2.6-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.8 MB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 16.8/16.8 MB 52.9 MB/s eta 0:00:00 Collecting pillow!=8.3.*,>=5.3.0 Downloading pillow-12.2.0-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (7.1 MB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 7.1/7.1 MB 86.1 MB/s eta 0:00:00 Collecting mpmath<1.4,>=1.1.0 Downloading mpmath-1.3.0-py3-none-any.whl (536 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 536.2/536.2 KB 56.6 MB/s eta 0:00:00 Collecting MarkupSafe>=2.0 Downloading https://download.pytorch.org/whl/markupsafe-3.0.3-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (20 kB) Installing collected packages: mpmath, typing-extensions, sympy, pillow, nvidia-nvtx-cu12, nvidia-nvjitlink-cu12, nvidia-nccl-cu12, nvidia-curand-cu12, nvidia-cufft-cu12, nvidia-cuda-runtime-cu12, nvidia-cuda-nvrtc-cu12, nvidia-cuda-cupti-cu12, nvidia-cublas-cu12, numpy, networkx, MarkupSafe, fsspec, filelock, triton, nvidia-cusparse-cu12, nvidia-cudnn-cu12, jinja2, nvidia-cusolver-cu12, torch, torchvision Successfully installed MarkupSafe-3.0.3 filelock-3.29.0 fsspec-2026.4.0 jinja2-3.1.6 mpmath-1.3.0 networkx-3.4.2 numpy-2.2.6 nvidia-cublas-cu12-12.4.5.8 nvidia-cuda-cupti-cu12-12.4.127 nvidia-cuda-nvrtc-cu12-12.4.127 nvidia-cuda-runtime-cu12-12.4.127 nvidia-cudnn-cu12-9.1.0.70 nvidia-cufft-cu12-11.2.1.3 nvidia-curand-cu12-10.3.5.147 nvidia-cusolver-cu12-11.6.1.9 nvidia-cusparse-cu12-12.3.1.170 nvidia-nccl-cu12-2.21.5 nvidia-nvjitlink-cu12-12.4.127 nvidia-nvtx-cu12-12.4.127 pillow-12.2.0 sympy-1.13.1 torch-2.5.1+cu124 torchvision-0.20.1+cu124 triton-3.1.0 typing-extensions-4.15.0 Looking in indexes: https://download.pytorch.org/whl/cu124 Collecting xformers==0.0.29.post1 Downloading https://download.pytorch.org/whl/cu124/xformers-0.0.29.post1-cp310-cp310-manylinux_2_28_x86_64.whl (44.1 MB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 44.1/44.1 MB 12.2 MB/s eta 0:00:00 Requirement already satisfied: numpy in ./venv/lib/python3.10/site-packages (from xformers==0.0.29.post1) (2.2.6) Requirement already satisfied: torch==2.5.1 in ./venv/lib/python3.10/site-packages (from xformers==0.0.29.post1) (2.5.1+cu124) Requirement already satisfied: filelock in ./venv/lib/python3.10/site-packages (from torch==2.5.1->xformers==0.0.29.post1) (3.29.0) Requirement already satisfied: nvidia-cuda-runtime-cu12==12.4.127 in ./venv/lib/python3.10/site-packages (from torch==2.5.1->xformers==0.0.29.post1) (12.4.127) Requirement already satisfied: nvidia-cusparse-cu12==12.3.1.170 in ./venv/lib/python3.10/site-packages (from torch==2.5.1->xformers==0.0.29.post1) (12.3.1.170) Requirement already satisfied: typing-extensions>=4.8.0 in ./venv/lib/python3.10/site-packages (from torch==2.5.1->xformers==0.0.29.post1) (4.15.0) Requirement already satisfied: nvidia-cusolver-cu12==11.6.1.9 in ./venv/lib/python3.10/site-packages (from torch==2.5.1->xformers==0.0.29.post1) (11.6.1.9) Requirement already satisfied: nvidia-nccl-cu12==2.21.5 in ./venv/lib/python3.10/site-packages (from torch==2.5.1->xformers==0.0.29.post1) (2.21.5) Requirement already satisfied: jinja2 in ./venv/lib/python3.10/site-packages (from torch==2.5.1->xformers==0.0.29.post1) (3.1.6) Requirement already satisfied: nvidia-nvjitlink-cu12==12.4.127 in ./venv/lib/python3.10/site-packages (from torch==2.5.1->xformers==0.0.29.post1) (12.4.127) Requirement already satisfied: sympy==1.13.1 in ./venv/lib/python3.10/site-packages (from torch==2.5.1->xformers==0.0.29.post1) (1.13.1) Requirement already satisfied: triton==3.1.0 in ./venv/lib/python3.10/site-packages (from torch==2.5.1->xformers==0.0.29.post1) (3.1.0) Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.4.127 in ./venv/lib/python3.10/site-packages (from torch==2.5.1->xformers==0.0.29.post1) (12.4.127) Requirement already satisfied: nvidia-cublas-cu12==12.4.5.8 in ./venv/lib/python3.10/site-packages (from torch==2.5.1->xformers==0.0.29.post1) (12.4.5.8) Requirement already satisfied: fsspec in ./venv/lib/python3.10/site-packages (from torch==2.5.1->xformers==0.0.29.post1) (2026.4.0) Requirement already satisfied: nvidia-cufft-cu12==11.2.1.3 in ./venv/lib/python3.10/site-packages (from torch==2.5.1->xformers==0.0.29.post1) (11.2.1.3) Requirement already satisfied: nvidia-nvtx-cu12==12.4.127 in ./venv/lib/python3.10/site-packages (from torch==2.5.1->xformers==0.0.29.post1) (12.4.127) Requirement already satisfied: networkx in ./venv/lib/python3.10/site-packages (from torch==2.5.1->xformers==0.0.29.post1) (3.4.2) Requirement already satisfied: nvidia-curand-cu12==10.3.5.147 in ./venv/lib/python3.10/site-packages (from torch==2.5.1->xformers==0.0.29.post1) (10.3.5.147) Requirement already satisfied: nvidia-cudnn-cu12==9.1.0.70 in ./venv/lib/python3.10/site-packages (from torch==2.5.1->xformers==0.0.29.post1) (9.1.0.70) Requirement already satisfied: nvidia-cuda-cupti-cu12==12.4.127 in ./venv/lib/python3.10/site-packages (from torch==2.5.1->xformers==0.0.29.post1) (12.4.127) Requirement already satisfied: mpmath<1.4,>=1.1.0 in ./venv/lib/python3.10/site-packages (from sympy==1.13.1->torch==2.5.1->xformers==0.0.29.post1) (1.3.0) Requirement already satisfied: MarkupSafe>=2.0 in ./venv/lib/python3.10/site-packages (from jinja2->torch==2.5.1->xformers==0.0.29.post1) (3.0.3) Installing collected packages: xformers Successfully installed xformers-0.0.29.post1 Obtaining file:///content/trainer/sd_scripts (from -r requirements.txt (line 45)) Preparing metadata (setup.py) ... done Collecting accelerate==0.33.0 Downloading accelerate-0.33.0-py3-none-any.whl (315 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 315.1/315.1 KB 6.0 MB/s eta 0:00:00 Collecting transformers==4.44.0 Downloading transformers-4.44.0-py3-none-any.whl (9.5 MB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 9.5/9.5 MB 85.7 MB/s eta 0:00:00 Collecting diffusers[torch]==0.25.0 Downloading diffusers-0.25.0-py3-none-any.whl (1.8 MB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 1.8/1.8 MB 63.3 MB/s eta 0:00:00 Collecting ftfy==6.1.1 Downloading ftfy-6.1.1-py3-none-any.whl (53 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 53.1/53.1 KB 7.8 MB/s eta 0:00:00 Collecting opencv-python==4.8.1.78 Downloading opencv_python-4.8.1.78-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (61.7 MB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 61.7/61.7 MB 12.2 MB/s eta 0:00:00 Collecting einops==0.7.0 Downloading einops-0.7.0-py3-none-any.whl (44 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 44.6/44.6 KB 7.2 MB/s eta 0:00:00 Collecting pytorch-lightning==1.9.0 Downloading pytorch_lightning-1.9.0-py3-none-any.whl (825 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 825.8/825.8 KB 69.7 MB/s eta 0:00:00 Collecting bitsandbytes==0.44.0 Downloading bitsandbytes-0.44.0-py3-none-manylinux_2_24_x86_64.whl (122.4 MB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 122.4/122.4 MB 8.0 MB/s eta 0:00:00 Collecting prodigyopt==1.0 Downloading prodigyopt-1.0-py3-none-any.whl (5.5 kB) Collecting lion-pytorch==0.0.6 Downloading lion_pytorch-0.0.6-py3-none-any.whl (4.2 kB) Collecting schedulefree==1.4 Downloading schedulefree-1.4.tar.gz (22 kB) Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done Collecting tensorboard Downloading tensorboard-2.20.0-py3-none-any.whl (5.5 MB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 5.5/5.5 MB 132.6 MB/s eta 0:00:00 Collecting safetensors==0.4.4 Downloading safetensors-0.4.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (435 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 435.5/435.5 KB 6.1 MB/s eta 0:00:00 Collecting altair==4.2.2 Downloading altair-4.2.2-py3-none-any.whl (813 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 813.6/813.6 KB 34.7 MB/s eta 0:00:00 Collecting easygui==0.98.3 Downloading easygui-0.98.3-py2.py3-none-any.whl (92 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 92.7/92.7 KB 16.0 MB/s eta 0:00:00 Collecting toml==0.10.2 Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB) Collecting voluptuous==0.13.1 Downloading voluptuous-0.13.1-py3-none-any.whl (29 kB) Collecting huggingface-hub==0.24.5 Downloading huggingface_hub-0.24.5-py3-none-any.whl (417 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 417.5/417.5 KB 41.5 MB/s eta 0:00:00 Collecting imagesize==1.4.1 Downloading imagesize-1.4.1-py2.py3-none-any.whl (8.8 kB) Collecting rich==13.7.0 Downloading rich-13.7.0-py3-none-any.whl (240 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 240.6/240.6 KB 35.8 MB/s eta 0:00:00 Collecting sentencepiece==0.2.0 Downloading sentencepiece-0.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 1.3/1.3 MB 75.2 MB/s eta 0:00:00 Collecting psutil Downloading psutil-7.2.2-cp36-abi3-manylinux2010_x86_64.manylinux_2_12_x86_64.manylinux_2_28_x86_64.whl (155 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 155.6/155.6 KB 25.1 MB/s eta 0:00:00 Requirement already satisfied: torch>=1.10.0 in ./venv/lib/python3.10/site-packages (from accelerate==0.33.0->-r requirements.txt (line 1)) (2.5.1+cu124) Collecting numpy<2.0.0,>=1.17 Downloading numpy-1.26.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.2 MB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 18.2/18.2 MB 79.1 MB/s eta 0:00:00 Collecting packaging>=20.0 Using cached packaging-26.2-py3-none-any.whl (100 kB) Collecting pyyaml Downloading pyyaml-6.0.3-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (770 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 770.3/770.3 KB 67.5 MB/s eta 0:00:00 Collecting requests Downloading requests-2.34.2-py3-none-any.whl (73 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 73.1/73.1 KB 12.4 MB/s eta 0:00:00 Collecting tokenizers<0.20,>=0.19 Downloading tokenizers-0.19.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 3.6/3.6 MB 92.9 MB/s eta 0:00:00 Collecting regex!=2019.12.17 Downloading regex-2026.5.9-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (794 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 794.1/794.1 KB 67.2 MB/s eta 0:00:00 Collecting tqdm>=4.27 Downloading tqdm-4.67.3-py3-none-any.whl (78 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 78.4/78.4 KB 11.3 MB/s eta 0:00:00 Requirement already satisfied: filelock in ./venv/lib/python3.10/site-packages (from transformers==4.44.0->-r requirements.txt (line 2)) (3.29.0) Requirement already satisfied: Pillow in ./venv/lib/python3.10/site-packages (from diffusers[torch]==0.25.0->-r requirements.txt (line 3)) (12.2.0) Collecting importlib-metadata Downloading importlib_metadata-9.0.0-py3-none-any.whl (27 kB) Collecting wcwidth>=0.2.5 Downloading wcwidth-0.7.0-py3-none-any.whl (110 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 110.8/110.8 KB 20.0 MB/s eta 0:00:00 Collecting lightning-utilities>=0.4.2 Downloading lightning_utilities-0.15.3-py3-none-any.whl (31 kB) Requirement already satisfied: fsspec[http]>2021.06.0 in ./venv/lib/python3.10/site-packages (from pytorch-lightning==1.9.0->-r requirements.txt (line 8)) (2026.4.0) Requirement already satisfied: typing-extensions>=4.0.0 in ./venv/lib/python3.10/site-packages (from pytorch-lightning==1.9.0->-r requirements.txt (line 8)) (4.15.0) Collecting torchmetrics>=0.7.0 Downloading torchmetrics-1.9.0-py3-none-any.whl (983 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 983.4/983.4 KB 82.4 MB/s eta 0:00:00 Collecting pandas>=0.18 Downloading pandas-2.3.3-cp310-cp310-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (12.8 MB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 12.8/12.8 MB 108.1 MB/s eta 0:00:00 Collecting entrypoints Downloading entrypoints-0.4-py3-none-any.whl (5.3 kB) Requirement already satisfied: jinja2 in ./venv/lib/python3.10/site-packages (from altair==4.2.2->-r requirements.txt (line 16)) (3.1.6) Collecting jsonschema>=3.0 Downloading jsonschema-4.26.0-py3-none-any.whl (90 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 90.6/90.6 KB 16.0 MB/s eta 0:00:00 Collecting toolz Downloading toolz-1.1.0-py3-none-any.whl (58 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 58.1/58.1 KB 9.9 MB/s eta 0:00:00 Collecting markdown-it-py>=2.2.0 Downloading markdown_it_py-4.2.0-py3-none-any.whl (91 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 91.7/91.7 KB 16.9 MB/s eta 0:00:00 Collecting pygments<3.0.0,>=2.13.0 Downloading pygments-2.20.0-py3-none-any.whl (1.2 MB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 1.2/1.2 MB 5.7 MB/s eta 0:00:00 Collecting tensorboard-data-server<0.8.0,>=0.7.0 Downloading tensorboard_data_server-0.7.2-py3-none-manylinux_2_31_x86_64.whl (6.6 MB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 6.6/6.6 MB 132.0 MB/s eta 0:00:00 Collecting werkzeug>=1.0.1 Downloading werkzeug-3.1.8-py3-none-any.whl (226 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 226.5/226.5 KB 35.6 MB/s eta 0:00:00 Collecting absl-py>=0.4 Downloading absl_py-2.4.0-py3-none-any.whl (135 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 135.8/135.8 KB 24.4 MB/s eta 0:00:00 Collecting grpcio>=1.48.2 Downloading grpcio-1.80.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (6.8 MB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 6.8/6.8 MB 113.8 MB/s eta 0:00:00 Collecting protobuf!=4.24.0,>=3.19.6 Downloading protobuf-7.35.0-cp310-abi3-manylinux2014_x86_64.whl (327 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 327.1/327.1 KB 37.8 MB/s eta 0:00:00 Collecting markdown>=2.6.8 Downloading markdown-3.10.2-py3-none-any.whl (108 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 108.2/108.2 KB 10.7 MB/s eta 0:00:00 Requirement already satisfied: setuptools>=41.0.0 in ./venv/lib/python3.10/site-packages (from tensorboard->-r requirements.txt (line 13)) (68.1.2) Collecting aiohttp!=4.0.0a0,!=4.0.0a1 Downloading aiohttp-3.13.5-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (1.7 MB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 1.7/1.7 MB 93.0 MB/s eta 0:00:00 Collecting attrs>=22.2.0 Downloading attrs-26.1.0-py3-none-any.whl (67 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 67.5/67.5 KB 11.1 MB/s eta 0:00:00 Collecting jsonschema-specifications>=2023.03.6 Downloading jsonschema_specifications-2025.9.1-py3-none-any.whl (18 kB) Collecting referencing>=0.28.4 Downloading referencing-0.37.0-py3-none-any.whl (26 kB) Collecting rpds-py>=0.25.0 Downloading rpds_py-0.30.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (390 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 390.5/390.5 KB 48.4 MB/s eta 0:00:00 Collecting mdurl~=0.1 Downloading mdurl-0.1.2-py3-none-any.whl (10.0 kB) Collecting python-dateutil>=2.8.2 Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 229.9/229.9 KB 37.8 MB/s eta 0:00:00 Collecting pytz>=2020.1 Downloading pytz-2026.2-py2.py3-none-any.whl (510 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 510.1/510.1 KB 43.8 MB/s eta 0:00:00 Collecting tzdata>=2022.7 Downloading tzdata-2026.2-py2.py3-none-any.whl (349 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 349.3/349.3 KB 48.0 MB/s eta 0:00:00 Requirement already satisfied: nvidia-cudnn-cu12==9.1.0.70 in ./venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate==0.33.0->-r requirements.txt (line 1)) (9.1.0.70) Requirement already satisfied: triton==3.1.0 in ./venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate==0.33.0->-r requirements.txt (line 1)) (3.1.0) Requirement already satisfied: sympy==1.13.1 in ./venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate==0.33.0->-r requirements.txt (line 1)) (1.13.1) Requirement already satisfied: nvidia-cuda-cupti-cu12==12.4.127 in ./venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate==0.33.0->-r requirements.txt (line 1)) (12.4.127) Requirement already satisfied: nvidia-nvjitlink-cu12==12.4.127 in ./venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate==0.33.0->-r requirements.txt (line 1)) (12.4.127) Requirement already satisfied: nvidia-cufft-cu12==11.2.1.3 in ./venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate==0.33.0->-r requirements.txt (line 1)) (11.2.1.3) Requirement already satisfied: nvidia-cuda-runtime-cu12==12.4.127 in ./venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate==0.33.0->-r requirements.txt (line 1)) (12.4.127) Requirement already satisfied: nvidia-cusparse-cu12==12.3.1.170 in ./venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate==0.33.0->-r requirements.txt (line 1)) (12.3.1.170) Requirement already satisfied: nvidia-curand-cu12==10.3.5.147 in ./venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate==0.33.0->-r requirements.txt (line 1)) (10.3.5.147) Requirement already satisfied: nvidia-nccl-cu12==2.21.5 in ./venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate==0.33.0->-r requirements.txt (line 1)) (2.21.5) Requirement already satisfied: nvidia-cublas-cu12==12.4.5.8 in ./venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate==0.33.0->-r requirements.txt (line 1)) (12.4.5.8) Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.4.127 in ./venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate==0.33.0->-r requirements.txt (line 1)) (12.4.127) Requirement already satisfied: networkx in ./venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate==0.33.0->-r requirements.txt (line 1)) (3.4.2) Requirement already satisfied: nvidia-cusolver-cu12==11.6.1.9 in ./venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate==0.33.0->-r requirements.txt (line 1)) (11.6.1.9) Requirement already satisfied: nvidia-nvtx-cu12==12.4.127 in ./venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate==0.33.0->-r requirements.txt (line 1)) (12.4.127) Requirement already satisfied: mpmath<1.4,>=1.1.0 in ./venv/lib/python3.10/site-packages (from sympy==1.13.1->torch>=1.10.0->accelerate==0.33.0->-r requirements.txt (line 1)) (1.3.0) Requirement already satisfied: markupsafe>=2.1.1 in ./venv/lib/python3.10/site-packages (from werkzeug>=1.0.1->tensorboard->-r requirements.txt (line 13)) (3.0.3) Collecting zipp>=3.20 Downloading zipp-4.1.0-py3-none-any.whl (10 kB) Collecting charset_normalizer<4,>=2 Downloading charset_normalizer-3.4.7-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (216 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 216.9/216.9 KB 32.3 MB/s eta 0:00:00 Collecting urllib3<3,>=1.26 Downloading urllib3-2.7.0-py3-none-any.whl (131 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 131.1/131.1 KB 23.2 MB/s eta 0:00:00 Collecting idna<4,>=2.5 Downloading idna-3.15-py3-none-any.whl (72 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 72.3/72.3 KB 12.7 MB/s eta 0:00:00 Collecting certifi>=2023.5.7 Downloading certifi-2026.5.20-py3-none-any.whl (134 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 134.1/134.1 KB 24.5 MB/s eta 0:00:00 Collecting aiohappyeyeballs>=2.5.0 Downloading aiohappyeyeballs-2.6.2-py3-none-any.whl (15 kB) Collecting frozenlist>=1.1.1 Downloading frozenlist-1.8.0-cp310-cp310-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl (219 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 219.5/219.5 KB 32.8 MB/s eta 0:00:00 Collecting multidict<7.0,>=4.5 Downloading multidict-6.7.1-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (243 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 243.3/243.3 KB 39.1 MB/s eta 0:00:00 Collecting propcache>=0.2.0 Downloading propcache-0.5.2-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (60 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 60.1/60.1 KB 8.3 MB/s eta 0:00:00 Collecting yarl<2.0,>=1.17.0 Downloading yarl-1.24.2-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (106 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 106.7/106.7 KB 14.4 MB/s eta 0:00:00 Collecting aiosignal>=1.4.0 Downloading aiosignal-1.4.0-py3-none-any.whl (7.5 kB) Collecting async-timeout<6.0,>=4.0 Downloading async_timeout-5.0.1-py3-none-any.whl (6.2 kB) Collecting six>=1.5 Downloading six-1.17.0-py2.py3-none-any.whl (11 kB) Building wheels for collected packages: schedulefree Building wheel for schedulefree (pyproject.toml) ... done Created wheel for schedulefree: filename=schedulefree-1.4-py3-none-any.whl size=39333 sha256=6de535190994da7772104109724f889c686c64f0d17e2ef9c949c9e55dbaed2f Stored in directory: /root/.cache/pip/wheels/04/4f/8e/15e0ac6c13041e94078d9e4f88d14c2cd00ec198bc0fc78e91 Successfully built schedulefree Installing collected packages: voluptuous, sentencepiece, pytz, library, easygui, zipp, werkzeug, wcwidth, urllib3, tzdata, tqdm, toolz, toml, tensorboard-data-server, six, schedulefree, safetensors, rpds-py, regex, pyyaml, pygments, psutil, protobuf, propcache, prodigyopt, packaging, numpy, multidict, mdurl, markdown, imagesize, idna, grpcio, frozenlist, entrypoints, einops, charset_normalizer, certifi, attrs, async-timeout, aiohappyeyeballs, absl-py, yarl, tensorboard, requests, referencing, python-dateutil, opencv-python, markdown-it-py, lightning-utilities, importlib-metadata, ftfy, aiosignal, rich, pandas, jsonschema-specifications, huggingface-hub, aiohttp, torchmetrics, tokenizers, lion-pytorch, jsonschema, diffusers, bitsandbytes, accelerate, transformers, pytorch-lightning, altair Running setup.py develop for library Attempting uninstall: numpy Found existing installation: numpy 2.2.6 Uninstalling numpy-2.2.6: Successfully uninstalled numpy-2.2.6 Successfully installed absl-py-2.4.0 accelerate-0.33.0 aiohappyeyeballs-2.6.2 aiohttp-3.13.5 aiosignal-1.4.0 altair-4.2.2 async-timeout-5.0.1 attrs-26.1.0 bitsandbytes-0.44.0 certifi-2026.5.20 charset_normalizer-3.4.7 diffusers-0.25.0 easygui-0.98.3 einops-0.7.0 entrypoints-0.4 frozenlist-1.8.0 ftfy-6.1.1 grpcio-1.80.0 huggingface-hub-0.24.5 idna-3.15 imagesize-1.4.1 importlib-metadata-9.0.0 jsonschema-4.26.0 jsonschema-specifications-2025.9.1 library-0.0.0 lightning-utilities-0.15.3 lion-pytorch-0.0.6 markdown-3.10.2 markdown-it-py-4.2.0 mdurl-0.1.2 multidict-6.7.1 numpy-1.26.4 opencv-python-4.8.1.78 packaging-26.2 pandas-2.3.3 prodigyopt-1.0 propcache-0.5.2 protobuf-7.35.0 psutil-7.2.2 pygments-2.20.0 python-dateutil-2.9.0.post0 pytorch-lightning-1.9.0 pytz-2026.2 pyyaml-6.0.3 referencing-0.37.0 regex-2026.5.9 requests-2.34.2 rich-13.7.0 rpds-py-0.30.0 safetensors-0.4.4 schedulefree-1.4 sentencepiece-0.2.0 six-1.17.0 tensorboard-2.20.0 tensorboard-data-server-0.7.2 tokenizers-0.19.1 toml-0.10.2 toolz-1.1.0 torchmetrics-1.9.0 tqdm-4.67.3 transformers-4.44.0 tzdata-2026.2 urllib3-2.7.0 voluptuous-0.13.1 wcwidth-0.7.0 werkzeug-3.1.8 yarl-1.24.2 zipp-4.1.0 Processing /content/trainer/custom_scheduler Preparing metadata (setup.py) ... done Using legacy 'setup.py install' for LoraEasyCustomOptimizer, since package 'wheel' is not installed. Installing collected packages: LoraEasyCustomOptimizer Running setup.py install for LoraEasyCustomOptimizer ... done Successfully installed LoraEasyCustomOptimizer-1.0.0 Collecting starlette Downloading starlette-1.0.0-py3-none-any.whl (72 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 72.7/72.7 KB 2.4 MB/s eta 0:00:00 Collecting uvicorn[standard] Downloading uvicorn-0.47.0-py3-none-any.whl (71 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 71.3/71.3 KB 9.1 MB/s eta 0:00:00 Requirement already satisfied: requests in ./venv/lib/python3.10/site-packages (from -r ../requirements.txt (line 3)) (2.34.2) Collecting dadaptation Downloading dadaptation-3.2.tar.gz (13 kB) Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done Collecting wandb Downloading wandb-0.27.0-py3-none-manylinux_2_28_x86_64.whl (27.3 MB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 27.3/27.3 MB 20.9 MB/s eta 0:00:00 Collecting pyngrok Downloading pyngrok-8.1.2-py3-none-any.whl (25 kB) Collecting pycloudflared Downloading pycloudflared-0.3.0-py3-none-any.whl (6.7 kB) Collecting scipy Downloading scipy-1.15.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (37.7 MB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 37.7/37.7 MB 22.1 MB/s eta 0:00:00 Collecting came-pytorch Downloading came_pytorch-0.1.3-py3-none-any.whl (6.2 kB) Collecting lycoris-lora==3.0.1.dev10 Downloading lycoris_lora-3.0.1.dev10.tar.gz (51 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 51.8/51.8 KB 9.1 MB/s eta 0:00:00 Preparing metadata (setup.py) ... done Collecting pytorch_optimizer==3.1.2 Downloading pytorch_optimizer-3.1.2-py3-none-any.whl (187 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 187.2/187.2 KB 26.7 MB/s eta 0:00:00 Collecting wheel Using cached wheel-0.47.0-py3-none-any.whl (32 kB) Requirement already satisfied: einops in ./venv/lib/python3.10/site-packages (from lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (0.7.0) Requirement already satisfied: toml in ./venv/lib/python3.10/site-packages (from lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (0.10.2) Requirement already satisfied: torch in ./venv/lib/python3.10/site-packages (from lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (2.5.1+cu124) Requirement already satisfied: tqdm in ./venv/lib/python3.10/site-packages (from lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (4.67.3) Requirement already satisfied: numpy in ./venv/lib/python3.10/site-packages (from pytorch_optimizer==3.1.2->-r ../requirements.txt (line 11)) (1.26.4) Requirement already satisfied: typing-extensions>=4.10.0 in ./venv/lib/python3.10/site-packages (from starlette->-r ../requirements.txt (line 1)) (4.15.0) Collecting anyio<5,>=3.6.2 Downloading anyio-4.13.0-py3-none-any.whl (114 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 114.4/114.4 KB 15.2 MB/s eta 0:00:00 Collecting h11>=0.8 Downloading h11-0.16.0-py3-none-any.whl (37 kB) Collecting click>=7.0 Downloading click-8.4.0-py3-none-any.whl (116 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 116.1/116.1 KB 17.9 MB/s eta 0:00:00 Requirement already satisfied: pyyaml>=5.1 in ./venv/lib/python3.10/site-packages (from uvicorn[standard]->-r ../requirements.txt (line 2)) (6.0.3) Collecting watchfiles>=0.20 Downloading watchfiles-1.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (457 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 457.1/457.1 KB 46.0 MB/s eta 0:00:00 Collecting uvloop>=0.15.1 Downloading uvloop-0.22.1-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (3.7 MB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 3.7/3.7 MB 81.4 MB/s eta 0:00:00 Collecting python-dotenv>=0.13 Downloading python_dotenv-1.2.2-py3-none-any.whl (22 kB) Collecting httptools>=0.6.3 Downloading httptools-0.7.1-cp310-cp310-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl (440 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 440.9/440.9 KB 50.6 MB/s eta 0:00:00 Collecting websockets>=10.4 Downloading websockets-16.0-cp310-cp310-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl (183 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 183.8/183.8 KB 22.4 MB/s eta 0:00:00 Requirement already satisfied: urllib3<3,>=1.26 in ./venv/lib/python3.10/site-packages (from requests->-r ../requirements.txt (line 3)) (2.7.0) Requirement already satisfied: idna<4,>=2.5 in ./venv/lib/python3.10/site-packages (from requests->-r ../requirements.txt (line 3)) (3.15) Requirement already satisfied: charset_normalizer<4,>=2 in ./venv/lib/python3.10/site-packages (from requests->-r ../requirements.txt (line 3)) (3.4.7) Requirement already satisfied: certifi>=2023.5.7 in ./venv/lib/python3.10/site-packages (from requests->-r ../requirements.txt (line 3)) (2026.5.20) Collecting platformdirs Downloading platformdirs-4.9.6-py3-none-any.whl (21 kB) Requirement already satisfied: protobuf!=5.28.0,!=5.29.0,<8,>4.21.0 in ./venv/lib/python3.10/site-packages (from wandb->-r ../requirements.txt (line 5)) (7.35.0) Collecting pydantic<3,>=2.6 Downloading pydantic-2.13.4-py3-none-any.whl (472 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 472.3/472.3 KB 12.9 MB/s eta 0:00:00 Collecting gitpython!=3.1.29,>=1.0.0 Downloading gitpython-3.1.50-py3-none-any.whl (212 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 212.5/212.5 KB 34.1 MB/s eta 0:00:00 Requirement already satisfied: packaging in ./venv/lib/python3.10/site-packages (from wandb->-r ../requirements.txt (line 5)) (26.2) Collecting sentry-sdk>=2.0.0 Downloading sentry_sdk-2.60.0-py3-none-any.whl (475 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 475.6/475.6 KB 56.0 MB/s eta 0:00:00 Collecting tomli Downloading tomli-2.4.1-py3-none-any.whl (14 kB) Collecting exceptiongroup>=1.0.2 Downloading exceptiongroup-1.3.1-py3-none-any.whl (16 kB) Collecting gitdb<5,>=4.0.1 Downloading gitdb-4.0.12-py3-none-any.whl (62 kB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 62.8/62.8 KB 11.6 MB/s eta 0:00:00 Collecting typing-inspection>=0.4.2 Downloading typing_inspection-0.4.2-py3-none-any.whl (14 kB) Collecting annotated-types>=0.6.0 Downloading annotated_types-0.7.0-py3-none-any.whl (13 kB) Collecting pydantic-core==2.46.4 Downloading pydantic_core-2.46.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.1 MB) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ 2.1/2.1 MB 115.4 MB/s eta 0:00:00 Requirement already satisfied: sympy==1.13.1 in ./venv/lib/python3.10/site-packages (from torch->lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (1.13.1) Requirement already satisfied: nvidia-cudnn-cu12==9.1.0.70 in ./venv/lib/python3.10/site-packages (from torch->lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (9.1.0.70) Requirement already satisfied: fsspec in ./venv/lib/python3.10/site-packages (from torch->lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (2026.4.0) Requirement already satisfied: nvidia-nccl-cu12==2.21.5 in ./venv/lib/python3.10/site-packages (from torch->lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (2.21.5) Requirement already satisfied: nvidia-cuda-runtime-cu12==12.4.127 in ./venv/lib/python3.10/site-packages (from torch->lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (12.4.127) Requirement already satisfied: nvidia-curand-cu12==10.3.5.147 in ./venv/lib/python3.10/site-packages (from torch->lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (10.3.5.147) Requirement already satisfied: nvidia-cusolver-cu12==11.6.1.9 in ./venv/lib/python3.10/site-packages (from torch->lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (11.6.1.9) Requirement already satisfied: nvidia-cufft-cu12==11.2.1.3 in ./venv/lib/python3.10/site-packages (from torch->lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (11.2.1.3) Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.4.127 in ./venv/lib/python3.10/site-packages (from torch->lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (12.4.127) Requirement already satisfied: networkx in ./venv/lib/python3.10/site-packages (from torch->lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (3.4.2) Requirement already satisfied: triton==3.1.0 in ./venv/lib/python3.10/site-packages (from torch->lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (3.1.0) Requirement already satisfied: nvidia-nvtx-cu12==12.4.127 in ./venv/lib/python3.10/site-packages (from torch->lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (12.4.127) Requirement already satisfied: nvidia-cublas-cu12==12.4.5.8 in ./venv/lib/python3.10/site-packages (from torch->lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (12.4.5.8) Requirement already satisfied: nvidia-cuda-cupti-cu12==12.4.127 in ./venv/lib/python3.10/site-packages (from torch->lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (12.4.127) Requirement already satisfied: jinja2 in ./venv/lib/python3.10/site-packages (from torch->lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (3.1.6) Requirement already satisfied: nvidia-nvjitlink-cu12==12.4.127 in ./venv/lib/python3.10/site-packages (from torch->lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (12.4.127) Requirement already satisfied: filelock in ./venv/lib/python3.10/site-packages (from torch->lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (3.29.0) Requirement already satisfied: nvidia-cusparse-cu12==12.3.1.170 in ./venv/lib/python3.10/site-packages (from torch->lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (12.3.1.170) Requirement already satisfied: mpmath<1.4,>=1.1.0 in ./venv/lib/python3.10/site-packages (from sympy==1.13.1->torch->lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (1.3.0) Collecting smmap<6,>=3.0.1 Downloading smmap-5.0.3-py3-none-any.whl (24 kB) Requirement already satisfied: MarkupSafe>=2.0 in ./venv/lib/python3.10/site-packages (from jinja2->torch->lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (3.0.3) Using legacy 'setup.py install' for lycoris-lora, since package 'wheel' is not installed. Building wheels for collected packages: dadaptation Building wheel for dadaptation (pyproject.toml) ... done Created wheel for dadaptation: filename=dadaptation-3.2-py3-none-any.whl size=23290 sha256=ce68082a1a3cfef5edad78f8f9b0efc2ebf6f3f8fc0b689961967d0701e881a8 Stored in directory: /root/.cache/pip/wheels/d0/03/6d/feba04df15ef39d9ac4e3504058ac2a88fb2ef9183ba92b111 Successfully built dadaptation Installing collected packages: wheel, websockets, uvloop, typing-inspection, tomli, smmap, sentry-sdk, scipy, python-dotenv, pyngrok, pydantic-core, platformdirs, httptools, h11, exceptiongroup, dadaptation, click, annotated-types, uvicorn, pydantic, pycloudflared, gitdb, anyio, watchfiles, starlette, gitpython, wandb, pytorch_optimizer, lycoris-lora, came-pytorch Running setup.py install for lycoris-lora ... done Successfully installed annotated-types-0.7.0 anyio-4.13.0 came-pytorch-0.1.3 click-8.4.0 dadaptation-3.2 exceptiongroup-1.3.1 gitdb-4.0.12 gitpython-3.1.50 h11-0.16.0 httptools-0.7.1 lycoris-lora-3.0.1.dev10 platformdirs-4.9.6 pycloudflared-0.3.0 pydantic-2.13.4 pydantic-core-2.46.4 pyngrok-8.1.2 python-dotenv-1.2.2 pytorch_optimizer-3.1.2 scipy-1.15.3 sentry-sdk-2.60.0 smmap-5.0.3 starlette-1.0.0 tomli-2.4.1 typing-inspection-0.4.2 uvicorn-0.47.0 uvloop-0.22.1 wandb-0.27.0 watchfiles-1.2.0 websockets-16.0 wheel-0.47.0 completed installing Found existing installation: rich 13.7.0 Uninstalling rich-13.7.0: Successfully uninstalled rich-13.7.0 โ
Installation finished in 309 seconds. ๐ Getting model... Download Results: gid |stat|avg speed |path/URI ======+====+===========+======================================================= f988f5|OK | 163MiB/s|//content/downloaded_model.safetensors Status Legend: (OK):download completed. Download Results: gid |stat|avg speed |path/URI ======+====+===========+======================================================= f2a2a9|OK | 237MiB/s|//content/sdxl_vae.safetensors Status Legend: (OK):download completed. ๐ Config saved to /content/drive/MyDrive/Loras/sed/training_config.toml ๐ Dataset config saved to /content/drive/MyDrive/Loras/sed/dataset_config.toml โญ Starting trainer... rich is not installed, using basic logging Loading settings from /content/drive/MyDrive/Loras/sed/training_config.toml... /content/drive/MyDrive/Loras/sed/training_config rich is not installed, using basic logging tokenizer_config.json: 100% 905/905 [00:00<00:00, 7.70MB/s] vocab.json: 961kB [00:00, 21.7MB/s] merges.txt: 525kB [00:00, 84.5MB/s] special_tokens_map.json: 100% 389/389 [00:00<00:00, 2.85MB/s] tokenizer.json: 2.22MB [00:00, 111MB/s] tokenizer_config.json: 100% 904/904 [00:00<00:00, 6.84MB/s] vocab.json: 862kB [00:00, 90.0MB/s] merges.txt: 525kB [00:00, 84.2MB/s] special_tokens_map.json: 100% 389/389 [00:00<00:00, 2.15MB/s] tokenizer.json: 2.22MB [00:00, 126MB/s] Loading dataset config from /content/drive/MyDrive/Loras/sed/dataset_config.toml prepare images. get image size from name of cache files 100% 404/404 [00:00<00:00, 116348.45it/s] set image size from cache files: 404/404 found directory /content/drive/MyDrive/Loras/sed/dataset contains 404 image files read caption: 76% 309/404 [01:48<00:34, 2.73it/s]neither caption file nor class tokens are found. use empty caption for /content/drive/MyDrive/Loras/sed/dataset/309.jpeg / ใญใฃใใทใงใณใใกใคใซใclass tokenใ่ฆใคใใใพใใใงใใใ็ฉบใฎใญใฃใใทใงใณใไฝฟ็จใใพใ: /content/drive/MyDrive/Loras/sed/dataset/309.jpeg read caption: 100% 404/404 [02:23<00:00, 2.81it/s] No caption file found for 1 images. Training will continue without captions for these images. If class token exists, it will be used. / 1ๆใฎ็ปๅใซใญใฃใใทใงใณใใกใคใซใ่ฆใคใใใพใใใงใใใใใใใฎ็ปๅใซใคใใฆใฏใญใฃใใทใงใณใชใใงๅญฆ็ฟใ็ถ่กใใพใใclass tokenใๅญๅจใใๅ ดๅใฏใใใไฝฟใใพใใ /content/drive/MyDrive/Loras/sed/dataset/309.jpeg 808 train images with repeating. 0 reg images. no regularization images / ๆญฃๅๅ็ปๅใ่ฆใคใใใพใใใงใใ [Dataset 0] batch_size: 4 resolution: (1024, 1024) enable_bucket: True network_multiplier: 1.0 min_bucket_reso: 256 max_bucket_reso: 4096 bucket_reso_steps: 64 bucket_no_upscale: False [Subset 0 of Dataset 0] image_dir: "/content/drive/MyDrive/Loras/sed/dataset" image_count: 404 num_repeats: 2 shuffle_caption: True keep_tokens: 1 keep_tokens_separator: caption_separator: , secondary_separator: None enable_wildcard: False caption_dropout_rate: 0.0 caption_dropout_every_n_epochs: 0 caption_tag_dropout_rate: 0.0 caption_prefix: None caption_suffix: None color_aug: False flip_aug: False face_crop_aug_range: None random_crop: False token_warmup_min: 1 token_warmup_step: 0 alpha_mask: False custom_attributes: {} is_reg: False class_tokens: None caption_extension: .txt [Dataset 0] loading image sizes. 100% 404/404 [00:00<00:00, 2484602.37it/s] make buckets number of images (including repeats) / ๅbucketใฎ็ปๅๆๆฐ๏ผ็นฐใ่ฟใๅๆฐใๅซใ๏ผ bucket 0: resolution (832, 1216), count: 2 bucket 1: resolution (1024, 1024), count: 804 bucket 2: resolution (1152, 896), count: 2 mean ar error (without repeats): 7.977811976167784e-05 preparing accelerator accelerator device: cuda loading model for process 0/1 load StableDiffusion checkpoint: /content/downloaded_model.safetensors building U-Net loading U-Net from checkpoint U-Net: <All keys matched successfully> building text encoders loading text encoders from checkpoint text encoder 1: <All keys matched successfully> text encoder 2: <All keys matched successfully> building VAE loading VAE from checkpoint VAE: <All keys matched successfully> load VAE: /content/sdxl_vae.safetensors additional VAE loaded Enable SDPA for U-Net import network module: networks.lora [Dataset 0] caching latents with caching strategy. caching latents... 100% 404/404 [02:47<00:00, 2.41it/s] create LoRA network. base dim (rank): 32, alpha: 16 neuron dropout: p=None, rank dropout: p=None, module dropout: p=None apply LoRA to Conv2d with kernel size (3,3). dim (rank): 32, alpha: 16.0 create LoRA for Text Encoder 1: create LoRA for Text Encoder 2: create LoRA for Text Encoder: 88 modules. create LoRA for U-Net: 788 modules. enable LoRA for text encoder: 88 modules enable LoRA for U-Net: 788 modules prepare optimizer, data loader etc. use 8-bit AdamW optimizer | {'weight_decay': 0.1, 'betas': [0.9, 0.99]} override steps. steps for 10 epochs is / ๆๅฎใจใใใฏใพใงใฎในใใใๆฐ: 2030 enable full fp16 training. running training / ๅญฆ็ฟ้ๅง num train images * repeats / ๅญฆ็ฟ็ปๅใฎๆฐร็นฐใ่ฟใๅๆฐ: 808 num reg images / ๆญฃๅๅ็ปๅใฎๆฐ: 0 num batches per epoch / 1epochใฎใใใๆฐ: 203 num epochs / epochๆฐ: 10 batch size per device / ใใใใตใคใบ: 4 gradient accumulation steps / ๅพ้
ใๅ่จใใในใใใๆฐ = 1 total optimization steps / ๅญฆ็ฟในใใใๆฐ: 2030 steps: 0% 0/2030 [00:00<?, ?it/s]unet dtype: torch.float16, device: cuda:0 text_encoder [0] dtype: torch.float16, device: cuda:0 text_encoder [1] dtype: torch.float16, device: cuda:0 epoch 1/10 epoch is incremented. current_epoch: 0, epoch: 1 steps: 10% 203/2030 [35:22<5:18:25, 10.46s/it, avr_loss=0.0933] saving checkpoint: /content/drive/MyDrive/Loras/sed/output/sed-01.safetensors epoch 2/10 epoch is incremented. current_epoch: 1, epoch: 2 steps: 20% 406/2030 [1:11:09<4:44:36, 10.51s/it, avr_loss=0.0928] saving checkpoint: /content/drive/MyDrive/Loras/sed/output/sed-02.safetensors epoch 3/10 epoch is incremented. current_epoch: 2, epoch: 3 steps: 30% 609/2030 [1:46:39<4:08:52, 10.51s/it, avr_loss=0.101] saving checkpoint: /content/drive/MyDrive/Loras/sed/output/sed-03.safetensors epoch 4/10 epoch is incremented. current_epoch: 3, epoch: 4 steps: 40% 812/2030 [2:22:13<3:33:20, 10.51s/it, avr_loss=0.103] saving checkpoint: /content/drive/MyDrive/Loras/sed/output/sed-04.safetensors epoch 5/10 epoch is incremented. current_epoch: 4, epoch: 5 steps: 50% 1015/2030 [2:57:44<2:57:44, 10.51s/it, avr_loss=0.0893] saving checkpoint: /content/drive/MyDrive/Loras/sed/output/sed-05.safetensors epoch 6/10 epoch is incremented. current_epoch: 5, epoch: 6 steps: 60% 1218/2030 [3:33:16<2:22:11, 10.51s/it, avr_loss=0.0881] saving checkpoint: /content/drive/MyDrive/Loras/sed/output/sed-06.safetensors epoch 7/10 epoch is incremented. current_epoch: 6, epoch: 7 steps: 70% 1421/2030 [4:08:48<1:46:37, 10.51s/it, avr_loss=0.0963] saving checkpoint: /content/drive/MyDrive/Loras/sed/output/sed-07.safetensors epoch 8/10 epoch is incremented. current_epoch: 7, epoch: 8 steps: 71% 1444/2030 [4:12:52<1:42:37, 10.51s/it, avr_loss=0.0973]
