FLUX.3 Klein — TorchAO Quantized

The whole pipeline is quantized with TorchAO so it runs locally on your GPU with reduced VRAM usage:

• Text encoder — quantized to 8-bit (int8, weight-only)
• Transformer — quantized to FP8 (dynamic activation + weight)

The weights are already quantized in this repo, so no quantization config is needed — just load and run:

import torch
from diffusers import FluxPipeline

pipeline = FluxPipeline.from_pretrained(
    "addlabsviral/flux-3-8bit",
    torch_dtype=torch.bfloat16,
)
pipeline.to("cuda")

image = pipeline("a cozy cabin in the woods at sunset").images[0]
image.save("output.png")

Sample output

Credits

All credits to the Flux team at Black Forest Labs for the base model.