<h1 id="yidhar-psd-layer-vae-an-rgba-autoencoder-for-editable-layered-psds">Yidhar psd-layer-vae — An RGBA Autoencoder for Editable Layered PSDs</h1>⚠️ Please read before downloading: This is a standalone VAE trained from scratch (f8c32: 1/8 resolution, 32-channel latent). It is not compatible with SD1.5 / SDXL / Flux or any other existing diffusion model, and cannot be used as a drop-in VAE replacement in WebUI or ComfyUI. Use the <code>psd_layer_vae</code> inference code included in the repository. It is intended for encoding/reconstructing layered images, RGBA data processing, and as a latent space for training layered generative models.<h2 id="what-it-is">What it is</h2>Each layer is encoded into its own latent. At decode time, all layers of the same artwork are decoded jointly with cross-layer attention, producing straight (non-premultiplied) RGBA. An HDR channel can reconstruct linear light brighter than SDR white. About 82M parameters in total (34.8M encoder + 47.2M decoder).<h2 id="key-strengths">Key strengths</h2>1. Clean transparent areas — no gray haze when stacking layers Existing RGBA VAEs often leave a faint alpha residue across the whole layer. Invisible on a single layer, but stack dozens of them and you get a gray veil; saved as an 8-bit PSD, it also breaks layer bounding boxes. This model outputs essentially zero in transparent regions:Average over 72 line-art layers (h3 vs Qwen-Image-2.1):<ul><li>Alpha soft IoU: 0.958 vs 0.578</li><li>Mean residual alpha in transparent areas: 0.02/255 vs 0.80/255</li></ul><img src="https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/b694abcd-8a8f-4303-92f1-63810f9782f4/original=true/b694abcd-8a8f-4303-92f1-63810f9782f4.jpeg" />2. Accurate thin lines, semi-transparent strokes and dense text Text on transparent layers: 42.06 dB PSNR (Qwen-Image-2.1: 29.82), alpha soft IoU 0.945 (vs 0.745).<img src="https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/29c37092-a480-49cf-9162-50ca923faa84/original=true/29c37092-a480-49cf-9162-50ca923faa84.jpeg" />3. Decode an entire PSD at once — faster, with less VRAM Compared with per-layer Qwen-Image-2.1 VAE decoding (H200, bf16): 2.5–3.7× faster decoding at about 1/3 the VRAM of its batched decode. At 1024² × 32 layers, Qwen's batched decode runs out of memory; this model completes normally.<img src="https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/830db50b-7643-423a-86c8-10250b06ae68/original=true/830db50b-7643-423a-86c8-10250b06ae68.jpeg" />4. Higher per-layer and composite fidelity On 20 held-out PSD windows (15.7 layers each on average): per-layer PSNR 53.96 vs 41.74; composite PSNR 37.00 vs 34.97.5. Strong on regular images too Leads both Qwen-Image-2.1 and FLUX.2 VAEs in PSNR / SSIM on every test set: ImageNet, LAION, photos, illustrations and danbooru.<img src="https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/87c40da6-143f-4b4c-85e2-6758aad5765b/original=true/87c40da6-143f-4b4c-85e2-6758aad5765b.jpeg" />6. HDR linear light support<ul><li>h3 (recommended) — for editing and exact per-layer reconstruction. Best pixel fidelity.</li><li>g3 — for sharper decoding in generation. About half the rFID of h3, ~0.3 dB lower PSNR.</li></ul>h3 and g3 use different latent spaces — do not mix their encoders and decoders.<h2 id="quick-start">Quick start</h2><pre><code>import torch
import psd_layer_vae as P

model = P.load('Yidhar/psd-layer-vae', 'h3') # or 'g3'
rgba = ... # (num_layers, 4, H, W), bottom to top, straight RGBA, sRGB, [0,1], H/W multiples of 8
latent, box = P.encode(model, rgba)
out = P.decode(model, latent, box)
image = P.composite(out['rgba'])
</code></pre>See the HuggingFace page for full usage (HDR, layer order, fp32, etc.).Known limitations<ul><li>Perceptual quality (LPIPS) still trails Qwen-Image-2.1 and FLUX.2. On rFID, g3 beats Qwen-Image-2.1 but not FLUX.2.</li><li>Joint decoding causes slight cross-layer crosstalk: after editing or deleting one layer and re-decoding, the other layers change by about 0.12/255 on average. If they must stay bit-identical, re-decode only the edited layer and keep the previous output for the rest.</li><li>Extremely bright HDR scenes (highlights above ~10 stops) still fall noticeably behind in some cases.</li><li>g3 details look realistic but are not identical to the source; use h3 when exact reconstruction matters.</li></ul><h3 id="thanks-to-comfy.org-and-others-for-providing-computing-resources.">Thanks to <a target="_blank" rel="ugc" href="http://comfy.org">comfy.org</a> and others for providing computing resources.</h3><h2 id="links">Links</h2>HuggingFace: <a target="_blank" rel="ugc" href="https://huggingface.co/Yidhar/psd-layer-vae">https://huggingface.co/Yidhar/psd-layer-vae</a>

Files with SHA256: c428ee53e476f4fc35989013aa9fbb88213500eede92b05ca489991e423699dc