Modificación Ggml-vulkan OpSUDot Poco f5
Como dice el título logre modificar el repo de leejet de difusión para agregar una ruta unmixed/mixed, esto hace que int dot se active, bajando el tiempo de 44s a 27, para bajarlo aún mas, modificamos el operando de la cuantizacion a OpSUDot pasando en anima-turbo de 44s por paso a solo 11/12s por paso, incluso logré correr Moss en un tiempo aceptable aunque por ahora en una resolución bajita, os dejo mi repo, https://github.com/frank982/ggml-adreno-vulkan/commits?author=frank982&since=2026-10-31&until=2026-11-05 , si le dais me gusta, enseño como instalar todo en termux y que paquetes son necesarios xD, es algo lioso pero bastante fácil, el único pillin es el paquete vulkan-loader-android, es el que hace que llmvpipe se quite, funcionando bien la adreno, entonces los custom drivers exponen esa ruta que nos hace falta y con el parche ya tenemos esos tiempos, e de decir que mi Poco es roteado y aligerado bastante con uad, solo paquetes imprescindibles, es la versión de 12 GB de ram $ ~/stable-repro-clean/build/bin/sd-cli --diffusion-model ./mossImageTurbo_v10_q5_1.gguf --llm ./MOSS/Qwen3.5-9B-UD-IQ2_XXS.gguf --vae ./MOSS/flux2-vae_small.safetensors --tokenizer ./MOSS/tokenizer.json --cfg-scale 1.0 -p "a lovely cat holding a sign says 'PocoF5'" -v -H 480 -W 360 --max-vram 9 --backend "te=cpu,tae=vulkan0,diffusion=vulkan0" -o ./test1.png --steps 4 --vae-tiling
[V] version: stable-diffusion.cpp version master-1028-37bc88b-1-ge239e4b+, commit e239e4b --- main.cpp:694
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = Turnip Adreno (TM) 725 (turnip Mesa driver) | uma: 1 | fp16: 1 | bf16: 0 | fp4: 0 | warp size: 128 | shared memory: 32768 | int dot: 1 | matrix cores: none
[V] System Info:
NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | MATMUL_INT8 = 1 | DOTPROD = 1 | OPENMP = 1 | REPACK = 1 | --- main.cpp:695
[V] using 8 threads for model loading --- model_loader.cpp:190
[I] loading diffusion model from './mossImageTurbo_v10_q5_1.gguf' --- diffusion_engine.cpp:692
[I] load ./mossImageTurbo_v10_q5_1.gguf using gguf format --- model_loader.cpp:216
[V] init from './mossImageTurbo_v10_q5_1.gguf' --- model_loader.cpp:262
[V] GGUF v3, tensor_count=378, metadata_kv_count=0 --- gguf_reader_ext.h:225
[I] loading llm from './MOSS/Qwen3.5-9B-UD-IQ2_XXS.gguf' --- diffusion_engine.cpp:749
[I] load ./MOSS/Qwen3.5-9B-UD-IQ2_XXS.gguf using gguf format --- model_loader.cpp:216
[V] init from './MOSS/Qwen3.5-9B-UD-IQ2_XXS.gguf' --- model_loader.cpp:262
[V] GGUF v3, tensor_count=427, metadata_kv_count=46 --- gguf_reader_ext.h:225
[I] loading vae from './MOSS/flux2-vae_small.safetensors' --- diffusion_engine.cpp:763
[I] load ./MOSS/flux2-vae_small.safetensors using safetensors format --- model_loader.cpp:222
[V] init from './MOSS/flux2-vae_small.safetensors', prefix = 'vae.' --- model_loader.cpp:290
[I] Version: MOSS-Image --- diffusion_engine.cpp:975
[I] auto-fit plan: --- backend_fit.cpp:323
[I] devices: --- backend_fit.cpp:324
[I] Vulkan0 Turnip Adreno (TM) 725 free 7249 MiB, budget 7249 MiB --- backend_fit.cpp:328
[I] RAM free 8054 MiB, params budget 6006 MiB --- backend_fit.cpp:334
[I] compute-device weight cache priority: diffusion > te > vae --- backend_fit.cpp:336
[I] components (params: compute device -> RAM -> other GPU -> disk): --- backend_fit.cpp:337
[I] DiT params 3932 MiB, compute reserve 2048 MiB -> compute Vulkan0, params Vulkan0 --- backend_fit.cpp:348
[I] Conditioner params 3032 MiB, compute reserve 2048 MiB -> compute CPU, params cpu --- backend_fit.cpp:348
[I] VAE params 237 MiB, compute reserve 1024 MiB -> compute Vulkan0, params Vulkan0 --- backend_fit.cpp:348
[I] auto-fit: --backend "te=cpu,tae=vulkan0,diffusion=vulkan0" --params-backend "te=cpu" --- backend_fit.cpp:477
[V] Initializing backend: Vulkan0 --- ggml_extend_backend.cpp:420
[V] using 8 threads for model loading --- model_loader.cpp:190
[I] Weight type stat: f32: 427 | q5_1: 341 | q8_0: 48 | q2_K: 4 | q3_K: 1 | q4_K: 24 | q5_K: 1 | iq2_xxs: 130 | iq3_xxs: 32 | iq3_s: 5 | iq2_s: 5 | i32: 1 | bf16: 37 --- diffusion_engine.cpp:1548
[I] Conditioner weight type stat: f32: 177 | q8_0: 48 | q2_K: 4 | q3_K: 1 | q4_K: 24 | q5_K: 1 | iq2_xxs: 130 | iq3_xxs: 32 | iq3_s: 5 | iq2_s: 5 --- diffusion_engine.cpp:1549
[I] Diffusion model weight type stat: q5_1: 341 | bf16: 37 --- diffusion_engine.cpp:1550
[I] VAE weight type stat: f32: 250 | i32: 1 --- diffusion_engine.cpp:1551
[V] Initializing backend: CPU --- ggml_extend_backend.cpp:420
[V] llm: num_layers = 32, vocab_size = 248320, hidden_size = 4096, intermediate_size = 12288 --- llm.hpp:446
[I] using external tokenizer (main): ./MOSS/tokenizer.json --- tokenizer_config.cpp:63
[V] moss_image: dim = 3072, ffn_dim = 8192, depth = 42, head_dim = 128 --- moss_image.hpp:52
[I] using VAE for encoding / decoding --- model_builders.cpp:210
[I] vae decoder: ch = 96 --- auto_encoder_kl.hpp:529
[I] flash attention auto: CPU -> on (CPU) --- flash_attn.cpp:66
[I] flash attention auto: Vulkan0 -> off (probe: normal 28.080 ms, flash 103.101 ms) --- flash_attn.cpp:95
[W] conv2d direct auto: Vulkan0 -> off (probe output check failed) --- conv_direct.cpp:79
[W] conv3d direct auto: Vulkan0 -> off (probe output check failed) --- conv_direct.cpp:79
[V] validating model metadata --- diffusion_engine.cpp:1307
[V] model metadata validated; weights will be prepared lazily --- diffusion_engine.cpp:1377
[I] total params memory size = 6421.20MB (VRAM 4055.73MB, RAM 2365.47MB): text_encoders 2365.47MB(RAM), diffusion_model 3933.53MB(VRAM), vae 122.20MB(VRAM), controlnet 0.00MB(N/A), extensions 0.00MB(N/A) --- diffusion_engine.cpp:1438
[I] running in FLOW mode --- diffusion_engine.cpp:1478
[W] align generation request up 360x480 to 368x480 (multiple=16) --- request.cpp:228
[I] generate_image 368x480 --- image.cpp:869
[V] using 8 threads for model loading --- model_loader.cpp:190
[I] get_sigmas with discrete scheduler --- denoiser.hpp:1129
[I] sampling using Euler method --- request.cpp:477
[V] parse '<|im_start|>user\na lovely cat holding a sign says 'PocoF5'<|im_end|>\n<|im_start|>assistant\n<think>\n' to [['<|im_start|>user\na lovely cat holding a sign says 'PocoF5'<|im_end|>\n<|im_start|>assistant\n<think>\n', 1], ] --- conditioner.hpp:2181
[V] split prompt "<|im_start|>user\na lovely cat holding a sign says 'PocoF5'<|im_end|>\n<|im_start|>assistant\n<think>\n" to 23 tokens ["<|im_start|>", "user", "Ċ", "a", "Ġlovely", "Ġcat", "Ġholding", "Ġa", "Ġsign", "Ġsays", "Ġ'", "P", "oco", "F", "5", "'", "<|im_end|>", "Ċ", "<|im_start|>", "assistant", "Ċ", "<think>", "Ċ", ] --- hf_tokenizer.cpp:739
[I] qwen3.5 build cached graph cut plan done (taking 4 ms) --- ggml_graph_cut.cpp:1042
[V] loading 426/427 tensors from ./MOSS/Qwen3.5-9B-UD-IQ2_XXS.gguf --- model_loader.cpp:1155 |##################################################| 426/426 - 1.65GB/s
[I] loading tensors completed, taking 1.40s (read: 1.32s, memcpy: 0.00s, convert: 0.00s, copy_to_backend: 0.00s) --- model_loader.cpp:1427
[V] model manager prepared params backend buffers (2365.47 MB, 426 tensors, 1 blocks, RAM) on CPU --- model_manager.cpp:494
[V] qwen3.5 compute buffer size: 7.06 MB(RAM) on CPU (peak across 1 segment) --- ggml_runner.cpp:1068 [V] computing condition graph completed, taking 5765 ms --- conditioner.hpp:3287
[I] get_learned_condition completed, taking 5.76s --- image.cpp:580
[I] generating image: 1/1 - seed 42 --- image.cpp:920
[I] MOSS-Image build cached graph cut plan done (taking 5 ms) --- ggml_graph_cut.cpp:1042
[V] loading 378/378 tensors from ./mossImageTurbo_v10_q5_1.gguf --- model_loader.cpp:1155
|##################################################| 378/378 - 1.13GB/s [I] loading tensors completed, taking 3.41s (read: 2.35s, memcpy: 0.00s, convert: 0.00s, copy_to_backend: 0.67s) --- model_loader.cpp:1427
[V] model manager prepared params backend buffers (3933.53 MB, 378 tensors, 4 blocks, VRAM) on Vulkan0 --- model_manager.cpp:494
[V] MOSS-Image compute buffer size: 1.41 MB(RAM) on CPU (peak across 1 segment) --- ggml_runner.cpp:1068
[V] MOSS-Image compute buffer size: 116.04 MB(VRAM) on Vulkan0 (peak across 1 segment) --- ggml_runner.cpp:1068
|==================================================| 4/4 - 19.34s/it
[I] sampling completed, taking 83.91s --- image.cpp:953
[I] generating 1 latent images completed, taking 83.91s --- image.cpp:967
[I] decoding 1 latents --- image.cpp:605
[V] VAE decode tile size: 256x256 pixels (16x16 latent) --- vae.hpp:348
[V] num tiles : 2, 2 --- tiling.cpp:178
[V] processing 4 tiles --- tiling.cpp:179
[I] vae build cached graph cut plan done (taking 0 ms) --- ggml_graph_cut.cpp:1042
[V] loading 140/251 tensors from ./MOSS/flux2-vae_small.safetensors --- model_loader.cpp:1155
|##################################################| 140/140 - 530.40MB/s
[I] loading tensors completed, taking 0.20s (read: 0.10s, memcpy: 0.00s, convert: 0.00s, copy_to_backend: 0.01s) --- model_loader.cpp:1427
[V] model manager prepared params backend buffers ( 54.48 MB, 140 tensors, 1 blocks, VRAM) on Vulkan0 --- model_manager.cpp:494
[V] vae compute buffer size: 264.77 MB(RAM) on CPU (peak across 1 segment) --- ggml_runner.cpp:1068
[V] vae compute buffer size: 312.00 MB(VRAM) on Vulkan0 (peak across 1 segment) --- ggml_runner.cpp:1068
|==================================================| 4/4 - 3.41s/it
[V] computing vae decode graph completed, taking 14.35s --- vae.hpp:384
[I] latent 1 decoded, taking 14.36s --- image.cpp:671
[I] decode_first_stage completed, taking 14.36s --- image.cpp:675
[I] generate_image completed in 104.04s --- image.cpp:1140
[V] model manager released params backend buffers (2365.47 MB, 426 tensors, 1 blocks, RAM) from CPU --- model_manager.cpp:1299
[V] model manager released params backend buffers (3988.02 MB, 518 tensors, 5 blocks, VRAM) from Vulkan0 --- model_manager.cpp:1299
[I] save result image 0 to './test1.png' (success) --- main.cpp:495
[I] 1/1 images saved --- main.cpp:571
~ $ mv test1.png /data/data/com.termux/files/home/storage/downloads
~ $. VULKANINFO
==========
Vulkan Instance Version: 1.3.366
Instance Extensions: count = 15
-------------------------------
VK_EXT_debug_report : extension revision 10
VK_EXT_debug_utils : extension revision 2
VK_EXT_headless_surface : extension revision 1
VK_EXT_surface_maintenance1 : extension revision 1
VK_EXT_swapchain_colorspace : extension revision 4
VK_GOOGLE_surfaceless_query : extension revision 2
VK_KHR_android_surface : extension revision 6
VK_KHR_device_group_creation : extension revision 1
VK_KHR_external_fence_capabilities : extension revision 1
VK_KHR_external_memory_capabilities : extension revision 1
VK_KHR_external_semaphore_capabilities : extension revision 1
VK_KHR_get_physical_device_properties2 : extension revision 2
VK_KHR_get_surface_capabilities2 : extension revision 1
VK_KHR_surface : extension revision 25
VK_KHR_surface_protected_capabilities : extension revision 1
Instance Layers:
----------------
Devices:
========
GPU0:
apiVersion = 1.4.363
driverVersion = 26.99.99
vendorID = 0x5143
deviceID = 0x7030002
deviceType = PHYSICAL_DEVICE_TYPE_INTEGRATED_GPU
deviceName = Turnip Adreno (TM) 725
driverID = DRIVER_ID_MESA_TURNIP
driverName = turnip Mesa driver
driverInfo = Mesa 27.0.0-devel (git-a79e3fc796)
conformanceVersion = 1.4.6.1
deviceUUID = a2ee5e4e-27bb-3d0b-c93e-caa4ce599044
driverUUID = 6e726152-67ae-6244-5908-10040553f8d2
time ./stable-repro-clean/build/bin/sd-cli --diffusion-model ./animaTurbo4SQuant_v10_q5_1.gguf --llm ./qwen_3_06b_base.safetensors --tae ./taew2_1.safetensors --max-vram 9 --backend "te=cpu,tae=cpu,diffusion=vulkan0" -t 8 -W 512 -H 512 --cfg-scale 1 --steps 4 -p "one girl" -o ./x.png
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = Turnip Adreno (TM) 725 (turnip Mesa driver) | uma: 1 | fp16: 1 | bf16: 0 | fp4: 0 | warp size: 128 | shared memory: 32768 | int dot: 1 | matrix cores: none
[I] loading diffusion model from './animaTurbo4SQuant_v10_q5_1.gguf' --- diffusion_engine.cpp:692
[I] load ./animaTurbo4SQuant_v10_q5_1.gguf using gguf format --- model_loader.cpp:216
[I] loading llm from './qwen_3_06b_base.safetensors' --- diffusion_engine.cpp:749
[I] load ./qwen_3_06b_base.safetensors using safetensors format --- model_loader.cpp:222
[I] loading tae from './taew2_1.safetensors' --- diffusion_engine.cpp:771
[I] load ./taew2_1.safetensors using safetensors format --- model_loader.cpp:222
[I] Version: Anima --- diffusion_engine.cpp:975
[I] auto-fit plan: --- backend_fit.cpp:323
[I] devices: --- backend_fit.cpp:324
[I] Vulkan0 Turnip Adreno (TM) 725 free 7495 MiB, budget 7495 MiB --- backend_fit.cpp:328
[I] RAM free 8328 MiB, params budget 6280 MiB --- backend_fit.cpp:334
[I] compute-device weight cache priority: diffusion > te > vae --- backend_fit.cpp:336
[I] components (params: compute device -> RAM -> other GPU -> disk): --- backend_fit.cpp:337
[I] DiT params 1517 MiB, compute reserve 2048 MiB -> compute Vulkan0, params Vulkan0 --- backend_fit.cpp:348
[I] Conditioner params 1136 MiB, compute reserve 2048 MiB -> compute CPU, params cpu --- backend_fit.cpp:348
[I] VAE params 21 MiB, compute reserve 1024 MiB -> compute CPU, params cpu --- backend_fit.cpp:348
[I] auto-fit: --backend "te=cpu,tae=cpu,diffusion=vulkan0" --params-backend "te=cpu,vae=cpu" --- backend_fit.cpp:477
[I] Weight type stat: f16: 148 | q5_1: 665 | bf16: 310 --- diffusion_engine.cpp:1548
[I] Conditioner weight type stat: bf16: 310 --- diffusion_engine.cpp:1549
[I] Diffusion model weight type stat: f16: 20 | q5_1: 665 --- diffusion_engine.cpp:1550
[I] VAE weight type stat: --- diffusion_engine.cpp:1551
[I] using TAE for encoding / decoding --- model_builders.cpp:207
[I] flash attention auto: CPU -> on (CPU) --- flash_attn.cpp:66
[I] flash attention auto: Vulkan0 -> off (probe: normal 29.136 ms, flash 101.681 ms) --- flash_attn.cpp:95
[W] conv2d direct auto: Vulkan0 -> off (probe output check failed) --- conv_direct.cpp:79
[W] conv3d direct auto: Vulkan0 -> off (probe output check failed) --- conv_direct.cpp:79
[I] conv2d direct auto: CPU -> off (no implicit GEMM path) --- conv_direct.cpp:90
[I] conv3d direct auto: CPU -> off (no implicit GEMM path) --- conv_direct.cpp:90
[I] total params memory size = 2676.65MB (VRAM 1518.06MB, RAM 1158.59MB): text_encoders 1137.00MB(RAM), diffusion_model 1518.06MB(VRAM), vae 21.59MB(RAM), controlnet 0.00MB(N/A), extensions 0.00MB(N/A) --- diffusion_engine.cpp:1438
[I] running in FLOW mode --- diffusion_engine.cpp:1478
[I] generate_image 512x512 --- image.cpp:869
[I] Using 'cosmos_reference' preset for reference images --- diffusion_engine.cpp:3103
[I] get_sigmas with discrete scheduler --- denoiser.hpp:1129
[I] sampling using Euler method --- request.cpp:477
[I] qwen3 build cached graph cut plan done (taking 1 ms) --- ggml_graph_cut.cpp:1042
|##################################################| 310/310 - 1.78GB/s
[I] loading tensors completed, taking 0.62s (read: 0.57s, memcpy: 0.00s, convert: 0.00s, copy_to_backend: 0.00s) --- model_loader.cpp:1427
[I] get_learned_condition completed, taking 0.82s --- image.cpp:580
[I] generating image: 1/1 - seed 42 --- image.cpp:920
[I] anima build cached graph cut plan done (taking 24 ms) --- ggml_graph_cut.cpp:1042
|##################################################| 685/685 - 1.47GB/s
[I] loading tensors completed, taking 1.01s (read: 0.70s, memcpy: 0.00s, convert: 0.00s, copy_to_backend: 0.14s) --- model_loader.cpp:1427
|==================================================| 4/4 - 12.08s/it
[I] sampling completed, taking 50.67s --- image.cpp:953
[I] generating 1 latent images completed, taking 50.67s --- image.cpp:967
[I] decoding 1 latents --- image.cpp:605
[I] taehv build cached graph cut plan done (taking 0 ms) --- ggml_graph_cut.cpp:1042
|##################################################| 64/64 - 92.50MB/s
[I] loading tensors completed, taking 0.20s (read: 0.07s, memcpy: 0.00s, convert: 0.00s, copy_to_backend: 0.00s) --- model_loader.cpp:1427
[I] latent 1 decoded, taking 6.97s --- image.cpp:671
[I] decode_first_stage completed, taking 6.97s --- image.cpp:675
[I] generate_image completed in 58.47s --- image.cpp:1140
[I] save result image 0 to './x.png' (success) --- main.cpp:495
[I] 1/1 images saved --- main.cpp:571
real 1m1.359s
user 0m57.604s
sys 0m15.005s