cstr/paddleocr-vl-1.6-GGUF

PaddleOCR-VL-1.6 — CrispEmbed GGUF

CrispEmbed-native GGUF quantizations of PaddlePaddle/PaddleOCR-VL-1.6.

Latest PaddleOCR-VL model with improved accuracy on OmniDocBench (96.3% SOTA). End-to-end VLM-based OCR: text recognition, table extraction, formula recognition, chart understanding. 109+ languages.

Files

File	Size	Description
paddleocr-vl-1.6-q4_k.gguf	1.3 GB	4-bit K-quant — smallest, lossy (see note)
paddleocr-vl-1.6-q8_0.gguf	1.4 GB	8-bit quantization — recommended
paddleocr-vl-1.6-f16.gguf	2.3 GB	fp16 reference

Quantization quality. This is a small 0.9B model, so 4-bit quantization is noticeably lossy. In testing, q8_0 transcribes fully and matches the fp16 reference, while q4_k tends to terminate early (e.g. it emitted only "The quick brown fox jumps" and stopped, dropping the rest of the line) — a flattened output distribution from 4-bit weights, not a bug. Behavior is identical on CPU and Metal. Use q8_0 for accurate OCR; reach for q4_k only when memory is tight and truncated/approximate output is acceptable.

Model

• Architecture: NaViT-style ViT (27L, 1152d) + ERNIE-4.5-0.3B LLM (18L, 1024d, 16/2 GQA, MRoPE, SwiGLU)
• Parameters: ~0.9B (same architecture as PaddleOCR-VL-0.9B, improved training)
• OmniDocBench: 96.3% (SOTA)
• Languages: 109+ (multilingual)
• License: Apache 2.0

Usage

./crispembed -m paddleocr-vl-1.6-q8_0.gguf --ocr document.png

License

Apache 2.0

Provenance and EU AI Act Art. 53 note

• Upstream model: PaddlePaddle/PaddleOCR-VL-1.6 — published by PaddlePaddle.
• Upstream licence: apache-2.0. This repository redistributes under the same terms; it grants no rights the upstream licence does not.
• What was done here: format conversion and/or quantisation only (GGUF). No training, no fine-tuning, no merging, no distillation, no change to architecture, vocabulary or capability. Only the numeric representation of the upstream weights differs.
• Training data: documented — where it is documented at all — by the upstream provider; see the upstream model card. No training data was used, added or selected by this repository. No training-content summary was found on the upstream model card at the time of writing; that documentation gap is upstream's and is not filled here.
• Provider status: under Regulation (EU) 2024/1689 the upstream authors remain the provider of this model. Converting the serialisation format does not make this repository the provider of a new general-purpose AI model, and no such claim is made. Questions about training content, copyright policy or model capability belong upstream.