RadixArk/Muse-Glimmer-q4-MLX

Muse Glimmer — MLX q4 (group size 64)

Overview

An Apple Silicon (MLX) checkpoint of Muse Glimmer, quantized directly from the vendor's BF16 HF export (the vendor BF16 HF export) with MLX affine 4-bit quantization at group size 64. The smallest and fastest of the Muse Glimmer MLX variants. Serve it with SGLang's MLX backend on a Mac with 48 GB of unified memory or more. Text only — the MLX backend has no vision path.

Model Specifications

• Source: the vendor BF16 HF export (BF16, text decoder; vision tower not included)
• Format: MLX-quantized safetensors, 3 shards (~15 GB, 4.50 bits/weight); tokenizer / config / chat template from the vendor HF export
• Quantization: MLX affine 4-bit, group size 64, uniform across the decoder
• Conventions: NeoX-style rotary (matches the 20260806 vendor HF convention), raw embedding table with runtime norm
• Architecture: 28B dense, 52 layers (13 full-attention NoPE + 39 sliding-window-2048), context 131,072
• Modality: text only

Evaluation Results

Measured through SGLang MLX on an M5 Pro (64 GB). GSM8K: 200 questions, no-thinking chat template, temperature 0. CIMemories: single trial, DeepSeek-R1-0528 judge (single-trial judge variance applies). References on the same machine and protocol: vendor llama.cpp GGUF q4km-gs128 scored 0.970 GSM8K / 8.27% violation / 68.4% coverage.

Benchmark	This checkpoint
GSM8K	0.975
CIMemories violation (lower better)	11.28%
CIMemories coverage (higher better)	89.0%

Note: this is the one Muse Glimmer MLX variant not derived from the vendor's calibrated quantization recipes; it trades a nonzero CIMemories violation rate for the highest coverage and the best speed. For the strictest privacy profile use the gs128 or dynamic mirror.

Performance (M5 Pro 64 GB, 1k-in/1k-out greedy)

Batch size	1	2	4	6	8
Decode tok/s (aggregate)	17.6	32.0	50.3	50.4	56.9

The fastest Apple Silicon configuration measured for this model on any runtime.

Serving with SGLang

SGLANG_USE_MLX=1 SGLANG_MLX_CACHE_LIMIT_GB=8 \
python -m sglang.launch_server \
  --model-path RadixArk/Muse-Glimmer-q4-MLX \
  --trust-remote-code \
  --reasoning-parser muse \
  --tool-call-parser muse \
  --disable-radix-cache \
  --mem-fraction-static 0.85 \
  --host 0.0.0.0 --port 30000

SGLANG_MLX_CACHE_LIMIT_GB=8 caps the MLX buffer cache (keeps the footprint within a 48 GB machine at no throughput cost). Keep --disable-radix-cache. See the Muse Glimmer page in the SGLang cookbook for the full deployment matrix.

Sibling checkpoints

• RadixArk/Muse-Glimmer-q4km-gs128-MLX — code-preserving mirror of the vendor GGUF (balanced pick)
• RadixArk/Muse-Glimmer-q4k-dynamic-MLX — mirror of the vendor's searched dynamic K-quant