Dove, a fictional character.

Trained on 34 Klein 9b generated images, from an initial JIB mix qwen generated image.

Trained with diffusion-pipe, this is a 128-rank lora trained on z-image (non-turbo) and qwen image 2512, using the ProdigyPlusScheduleFree optimizer (https://github.com/tdrussell/diffusion-pipe/pull/483), and masked training with backgrounds masked to 10%.

The model was evaluated every 340 steps (10 repeats of each training image) using FaceNet512 to evaluate the face similarity, using the same settings, seed, etc for inference, calculating the average distance to a subset of 22 of the training images.

The attached training data zip includes: the diffusion pipe config files; the training images, prompts, and masks; the validation images (subset of training images); and the output of MirrorMetrics evaluation of the chosen epoch and some additional candidate epochs vs. the validation set.

z-image

Epoch 19 is overtrained, resulting in some visual artifacts and degraded quality, but due to the overtraining can produce a more accurate face, mathematically.

epoch19:

epoch10:

Evaluation:

• For average distances, lower is better. Distance under 0.3 is considered a face match.

• Matching percentage is the percent of the 22 validation images that the generated face was under 0.3 distance from.

• Due to the large size and variety of the validation set, face match values are lower than I would typically consider good.

• epoch 10 (3,400 steps)

  • z-image (non-turbo) inference

    • Average distance: 0.385

    • Matching percentage: 14%

  • z-image-turbo inference

    • Average distance: 378

    • Matching percentage: 18%

• epoch 19 (6,460 steps)

  • z-image (non-turbo) inference

    • Average distance: 0.297

    • Matching percentage: 36%

  • z-image-turbo inference

    • Average distance: 0.357

    • Matching percentage: 45%

Evaluation using MirrorMetrics (html output included in training data attachment)

Training command:

PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True NCCL_P2P_DISABLE="1" NCCL_IB_DISABLE="1" deepspeed \
  --num_gpus=1 train.py \ 
  --deepspeed \ 
  --regenerate_cache \
  --config /home/user/diffusion-pipe-config.toml