RedHatAI/GLM-5.2-speculator.dspark

This is a preliminary (and subject to change) DSpark speculator model for nvidia/GLM-5.2-NVFP4.

It was trained using the Speculators library on the mgoin/open-perfectblend-glm5.2-regen dataset.

Note:

It was validated on Nvidia B200, other hardware validation pending.

Training Details

Launch training

scripts/train.py \
  --verifier-name-or-path RedHatAI/GLM-5.2-NVFP4-FP8 \
  --data-path ./output/dspark_glm52 \
  --vllm-endpoint http://localhost:8000/v1 \
  --save-path ./output/dspark_glm52/checkpoints \
  --epochs 1 \
  --lr 3e-4 \
  --total-seq-len 8192 \
  --speculator-type dspark \
  --block-size 8 \
  --max-anchors 1024 \
  --num-layers 3 \
  --target-layer-ids 2 20 39 58 75 \
  --markov-rank 256 \
  --markov-head-type vanilla \
  --enable-confidence-head \
  --confidence-head-with-markov \
  --loss-fn '{"ce": 0.1, "tv": 0.9}' \
  --confidence-head-alpha 1.0 \
  --fsdp-shard \
  --logger tensorboard \
  --log-dir ./output/dspark_glm52/logs \
  --on-missing generate \
  --on-generate delete \
  --checkpoint-freq 0.01 \
  --request-timeout 60 \
  --max-retries 2

Deployment

Deploy with vLLM using the speculator as a draft model.

vllm serve nvidia/GLM-5.2-NVFP4 \
  --max-model-len 20480 \
  --spec-model RedHatAI/GLM-5.2-speculator.dspark \
  --spec-method dspark \
  --spec-tokens 7 \
  -tp 8

Preliminary Evaluations

Per-Position Acceptance Rate

Dataset	Pos 0	Pos 1	Pos 2	Pos 3	Pos 4	Pos 5	Pos 6	Avg. Length
HumanEval	77.8%	59.1%	44.6%	33.9%	26.1%	20.3%	15.8%	3.78
math_reasoning	87.5%	76.6%	67.6%	59.2%	51.8%	45.1%	39.5%	5.27
qa	70.7%	48.5%	33.7%	23.7%	17.1%	11.9%	8.4%	3.14
question	73.7%	52.6%	38.1%	28.5%	21.6%	16.7%	13.2%	3.44
rag	78.5%	61.3%	48.7%	38.9%	30.7%	24.2%	19.2%	4.01
summarization	77.9%	58.0%	43.0%	31.7%	22.8%	16.2%	11.4%	3.61
tool_call	71.9%	51.1%	37.1%	27.6%	21.4%	16.8%	13.2%	3.39
translation	76.8%	57.4%	43.5%	33.6%	26.1%	20.0%	15.1%	3.72
writing	74.0%	52.2%	38.0%	28.1%	21.3%	16.7%	13.1%	3.44