latentdivergence/flux2-klein-edit-npu-8gen3

FLUX.2 [klein] 4B Edit -- Hexagon NPU (Snapdragon 8 Gen 3 (and other Hexagon v75 chips))

Instruction-based photo editing ("make it night", "add snow", "turn it into a watercolour") with FLUX.2 [klein] 4B running on the Qualcomm Hexagon NPU, precompiled for Snapdragon 8 Gen 3 (and other Hexagon v75 chips) (QNN context binaries, QAIRT 2.45). Built for the Latent Studio Android app ("Edit with words"); the prompt encoder lives in latentdivergence/flux2-klein-edit-text.

Resolution	512 x 512 (64 text tokens + 1024 latent + 1024 reference tokens)
Steps	4 (flow-matching Euler, distilled, no guidance)
Transformer	3 graphs, w8a16 (int8 weights, 16-bit activations), calibrated on real edit trajectories
VAE	encoder and decoder, fp16
AI Hub timing (Samsung Galaxy S24)	p1 524 ms + p2 1112 ms + p3 1148 ms = 2.78 s per step, about 11.1 s of transformer per edit; measured on a 16 GB Snapdragon 8 Gen 3 phone: 17-18.6 s per edit with a new instruction, 12.5 s for a retry with the same words, 26 s cold
Fidelity vs fp32 (per graph, AI Hub)	p1 image 0.99992 / text 0.99357, p2 image 0.99919, p3 output 0.9831 (min over held-out real edit calls)

Files

• transformer_p1/ embedders + 5 double-stream blocks; transformer_p2/ single blocks 0-9; transformer_p3/ single blocks 10-19 + output. Each is an ONNX EPContext wrapper (model.onnx) beside its QNN context binary (model.bin); fp32 inputs/outputs.
• vae_encoder/ image [1,3,512,512] in [-1,1] -> latent [1,128,32,32] (patchified, batch-norm normalised); vae_decoder/ the inverse.
• tables_512.npz per-step sigmas and the timestep-only modulation vectors (the graphs take them as inputs).

Host maths (what the app does)

sequence = [text | noisy latent | reference latent]; reference tokens carry time coordinate 10 (baked into the RoPE). The text residual stream is carried at 1/8: divide the Qwen embeddings and the two text gates of mod_txt (chunks 2 and 5 of 6 x 3072) by 8. Per step: p1 -> p2 -> p3 -> velocity; latent += (sigma[i+1] - sigma[i]) * velocity.

Licence

Apache-2.0, as the original model. See LICENSE.txt for the modifications made.