LightOnOCR-3-0.8B GGUF

GGUF conversion of lightonai/LightOnOCR-3-0.8B, made for local inference with llama.cpp. The original model is licensed under Apache 2.0.

File Size Purpose
model-Q4_K_M.gguf 542 MB Default quantized language model with MTP
mmproj-F16.gguf 205 MB Vision encoder and projector, required for images

llama.cpp 0.6.0 or newer can download the vision file automatically and enable MTP decoding:

llama-server -hf staghado/LightOnOCR-3-0.8B-GGUF:Q4_K_M \
  --spec-type draft-mtp \
  --chat-template-kwargs '{"enable_thinking":false}'

Send an image with an empty text prompt for transcription. The model also supports the grounding prompt for boxes and labels, as described in the original model card.

The Q4_K_M model and matching projector were smoke tested with llama.cpp on the receipt image linked from the original model card. This is a functionality check, not a quality benchmark.

Downloads last month
616
GGUF
Model size
0.8B params
Architecture
qwen35
Hardware compatibility
Log In to add your hardware

4-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for staghado/LightOnOCR-3-0.8B-GGUF

Quantized
(18)
this model