Instructions to use MonsterMMORPG/Wan_GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use MonsterMMORPG/Wan_GGUF with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("MonsterMMORPG/Wan_GGUF", dtype=torch.bfloat16, device_map="cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
Download ACEStep_1_5_XL_SFT_INT8_ConvRot/README.md from MonsterMMORPG/Wan_GGUF: direct link, hf CLI and curl.
- Browser
- Download file 3.73 kB
-
https://huggingface.co/MonsterMMORPG/Wan_GGUF/resolve/main/ACEStep_1_5_XL_SFT_INT8_ConvRot/README.md
- Command line
-
hf download hf://MonsterMMORPG/Wan_GGUF/ACEStep_1_5_XL_SFT_INT8_ConvRot/README.md
-
curl -L -o README.md https://huggingface.co/MonsterMMORPG/Wan_GGUF/resolve/main/ACEStep_1_5_XL_SFT_INT8_ConvRot/README.md
license: mit
pipeline_tag: text-to-audio
tags:
- audio
- music
- text2music
- int8
- quantized
- convrot
ACE-Step 1.5 XL SFT - INT8 ConvRot (prequantized)
Prequantized INT8 ConvRot build of the ACE-Step 1.5 XL SFT DiT, packaged for ACE-Step Premium.
The app loads this folder when the DiT quantization is set to INT8 ConvRot (best quality or fastest).
It is downloaded automatically the first time it is needed, or with Windows_Download_All_Models.bat
(HF_model_downloader.py --model ACEStep_1_5_XL_SFT_INT8_ConvRot).
Source model: ACEStep_1_5_XL_SFT_BF16 (ACE-Step 1.5, MIT license). Upstream project:
ACE-Step 1.5 |
Hugging Face |
Tech Report
Format
- The 288 transformer-block projections of the DiT decoder (self-attention q/k/v/o, cross-attention q/o and MLP gate/up/down in all 32 layers) are INT8: each row is rotated by a regular 256x256 Hadamard matrix per 256-input group (ConvRot) and quantized with one FP32 scale per output row (MSE-optimal clipping).
- Every other tensor (cross-attention k/v, embeddings, norms, condition encoder, ...) is the unchanged BF16 source tensor.
- Weight error vs BF16: mean 0.87%, max
1.44% (relative L2 per layer), min cosine 0.99995.
Per-layer numbers are in
int8_convrot_report.json. - File size 5.66 GiB (BF16 source 9.29 GiB). Tensors use the
int8_tensorwise+convrotlayer markers (.comfy_quant,_quantization_metadata) of ComfyUI's INT8 ConvRot convention; this folder is packaged and tested for ACE-Step Premium only.
Runtime (ACE-Step Premium)
- NVIDIA SM80+ GPUs (RTX 30/40/50, A100, H100, ...): W8A8 with Triton kernels - fused rotate + quantize of the activations, INT8 tensor-core GEMM with INT32 accumulation, dynamic activation scales per (row, 256 group) for best quality or per row for fastest; q/k/v and gate/up run as single fused GEMMs, SwiGLU is fused into the down-projection quantizer, and the residual stream and timestep MLP run in FP32.
- Other GPUs: exact weight-only fallback (same weights, BF16/FP16 matmul).
Measured accuracy and speed (RTX 5090)
Relative L2 error of the DiT velocity output vs an FP32 run on the same captured inputs, averaged over every
denoising step of three real generations (60 s instrumental, 120 s pop vocal, 240 s orchestral vocal).
The guided error is the error of uncond + 7 * (cond - uncond); the guidance amplifies the small difference between the conditional and unconditional predictions, so it is the strictest number.
| Path | Conditional velocity error | Guided velocity error (CFG 7) | DiT step time, ms (60 s / 120 s / 240 s song) |
|---|---|---|---|
| BF16, original path | 1.56% | 9.62% | 96.8 / 180.1 / 372.3 |
| BF16, new FP32-residual fused path | 0.76% | 4.10% | 75.2 / 142.1 / 290.5 |
| INT8 ConvRot, best quality | 1.05% | 5.51% | 56.0 / 87.5 / 186.8 |
| INT8 ConvRot, fastest | 1.15% | 6.51% | 49.6 / 73.9 / 152.8 |
INT8 ConvRot is more accurate than the original BF16 inference path, because the INT8 runtime keeps the residual stream in FP32; the new BF16 path (same FP32 residual) remains the most accurate option.
End to end (50 steps + CFG, steady state, BF16 original path -> INT8 ConvRot best quality): DiT time 1.92-2.06x faster, total generation time 1.67-1.83x faster, peak allocated VRAM 11.88 GiB -> 8.34 GiB (DiT + VAE + text encoder, no LM). The first generation after loading also tunes the INT8 GEMM kernels for the new shapes (a few seconds once; Triton caches the result).