Diffusers Requires New Python - Now we lost Xformers

I kept using diffusers with python 3.10 for as long as I could bypassing the checks for python but kept noticing more and more issues.

I cleared my 180k file VENV that worked for 20 different training programs over the last two years and did a full update.

Here are the issues. (For windows users of course Linux your fine)

• Bitsandbytes - can be compiled for Cuda 13.3 and Pytorch - You have to do this yourself

• Xformers - Can be compiled but requires MSLK to work with python 3.13 - this was not the case for 3.10

This is a major speed issue as xformers was a bridge for SDPA, Cutlass and a lot of backend attention GEMMS

I did manage to make CMAKE files for MSLK that work with windows but I have been unable to fully build it. Despite hours of work I am stuck at:

2 errors detected in the compilation of "MSLK-main/csrc/conv/cutlass/f8f8bf16_conv/f8f8bf16_conv_128x128x128_2x1x1.cu".
f8f8bf16_conv_128x128x128_2x1x1.cu
ninja: build stopped: subcommand failed.

I think a lot of the complaints I have been getting about model speed is do to me having 3.10 and working XFORMERS, and now I do not I see what the issue is.

Note: You can do

 set MSLK_PYTHON_ONLY=1

then in the MSLK git clone

pip install -e .

However this does not link the cuda files, it just makes a dummy file and you still end up with:

WARNING[XFORMERS]: the 'mslk' package is not installed, so the attention ops are unavailable.
  xformers.ops.memory_efficient_attention and the related symbols will not exist.
  mslk is a dependency of this part of xFormers.
WARNING[XFORMERS]: the 'mslk' package is not installed, so the attention ops are unavailable.
  xformers.ops.memory_efficient_attention and the related symbols will not exist.
  mslk is a dependency of this part of xFormers.

Despite looking like xformers is loading

[INFO] pytorch version: 2.14.1+cu132
[INFO] MSLK running in python-only mode — skipping native library loading
[INFO] xformers version: 0.0.35

Without Flash attention, SDPA, Cutlass, Xformers, training and interface falls back to Pyattention which can double or triple training times.

While I have compiled flash attention for windows before it took so much time to custom build I really do not want to have to do that again.

While the training time looks ok, and triton is working I wonder about what attention its falling back too: