Yaso76/Kolibri-1-GGUF

Kolibri-1 GGUF (Q8_0, mainline llama.cpp conversion)

Aleph Alpha's Kolibri-1 as a Q8_0 GGUF. It was made with the official llama.cpp converter from the BF16 weights. It loads in mainline llama.cpp b11709 or later without any patch and without --override-kv.

Unofficial. This conversion was not made, reviewed or endorsed by Aleph Alpha. For everything about the model itself, see the original model card.

Files

File	Size (bytes)	SHA256
Kolibri-1-Q8_0-00001-of-00005.gguf	19,949,189,408	2fb1ea7c3d7f4ec1373c627422da4a7cb3990c2970135d2a2c42e860b3b76497
Kolibri-1-Q8_0-00002-of-00005.gguf	19,784,216,032	f656c3c1ccd1e37773c7d48f104786d0e0a1888d5c6de5687e335ebd2e68d099
Kolibri-1-Q8_0-00003-of-00005.gguf	19,784,216,032	da78410763dfb4c1b9d92b05ce5e1b1b3f1fe46cf95dcd361772ed2be6a75605
Kolibri-1-Q8_0-00004-of-00005.gguf	19,784,216,032	0eb02404af9076c08b043968879864f68987724beea4ac987b3d1a3d7f45e638
Kolibri-1-Q8_0-00005-of-00005.gguf	3,833,556,576	2d5cd17a5fee75dc06fc86df2568d46c97f187da705a8de707c87d34b672cb81

Total 83.1 GB (77.42 GiB, 8.51 bits per weight). The same hashes are in SHA256SUMS.

Point llama.cpp at the first file. It loads the other parts automatically.

Provenance

• Source: Aleph-Alpha/Kolibri-1-BF16 at commit 38a4e5bc3c3192d1f458d043ac1a063a6380fdbd. All 32 safetensors were verified against their LFS SHA256 before conversion.
• llama.cpp: release tag b11889, commit d854d98bae598388ecea1ed84396587e5edc326d. Source and binaries (prebuilt Vulkan) are the same commit. Kolibri-1 support (kolibri1) came with PR #30673.
• Converter environment: Python 3.13.16, torch 2.11.0+cpu, transformers 4.57.6, numpy 2.2.6, installed from llama.cpp's requirements/requirements-convert_hf_to_gguf.txt. The gguf-py package came from the same checkout.
• Date / hardware: 2026-11-08. AMD Ryzen AI Max+ 395 with Radeon 8060S and 128 GB unified memory, Fedora 44.

Exact commands:

# 1. HF safetensors -> BF16 GGUF (run inside the llama.cpp b11889 checkout)
python convert_hf_to_gguf.py Kolibri-1-BF16/ --outtype bf16 --outfile Kolibri-1-BF16.gguf

# 2. BF16 GGUF -> Q8_0
llama-quantize Kolibri-1-BF16.gguf Kolibri-1-Q8_0.gguf Q8_0 32

# 3. Split into parts of at most 20G
llama-gguf-split --split --split-max-size 20G Kolibri-1-Q8_0.gguf Kolibri-1-Q8_0

Tensor types: 502 tensors are Q8_0. These are the attention q/k/v/output, the routed and shared expert FFNs, token_embd and output. 401 tensors stay F32: all norms, the router ffn_gate_inp and exp_probs_b.bias.

Key metadata, as written by the mainline converter: general.architecture = kolibri1, tokenizer.ggml.pre = qwen2, kolibri1.expert_gating_func = 2, kolibri1.attention.sliding_window = 513, kolibri1.attention.sliding_window_pattern as a 50-entry array (4 sliding : 1 full), kolibri1.context_length = 262144. The chat template is embedded.

Compatibility

• llama.cpp b11709 or later: loads as-is. Tested with b11889, Vulkan backend, on Strix Halo (Radeon 8060S, RADV).
• Many Kolibri-1 GGUFs on the Hub were made with the pre-merge patch (tokenizer.ggml.pre = kolibri1, gating mode 5). Mainline only accepts those with --override-kv (see discussion #3). This file does not need that.
• Ollama, LM Studio and other apps depend on the llama.cpp version they bundle. They need b11709 or later. Not tested.

Usage

llama-server -m Kolibri-1-Q8_0-00001-of-00005.gguf \
  -ngl 999 -fa on --jinja -c 32768 \
  --temp 1.0 --top-p 0.97 --top-k 128

• Sampling: temperature 1.0, top_p 0.97, top_k 128, as given in the original generation_config.json.

• Reasoning effort: set it through the chat template variable reasoning_effort, using chat_template_kwargs in the OpenAI-compatible API:

  {"messages": [...], "chat_template_kwargs": {"reasoning_effort": "none"}}
  

  The values are none, low (also minimal), medium and high (also xhigh, max). If you leave it out, high applies. none (or enable_thinking: false) turns reasoning off. llama-server returns the reasoning separately in reasoning_content.

• Tool calling: Hermes style (<tool_call>{"name": …, "arguments": …}</tool_call>). With --jinja, llama-server returns structured tool_calls.

• Context: the GGUF stores context_length = 262144. The original card states 1,048,576 tokens and recommends at most 262,144 for serving. On a 128 GB machine, KV-cache memory limits the context you can actually use. The tests below ran with -c 32768 and a q8_0 KV cache.

Validation

All tests ran with llama.cpp b11889 (Vulkan) on the hardware listed above, using the split files exactly as uploaded.

Smoke test (llama-server, --jinja, sampling as above). All six requests ended at the stop token, and none looped:

#	Request	Result
1	German, effort none: explain a reverse proxy in five sentences	Correct and fluent. No reasoning content.
2	German, effort low: rewrite a formal notice in a casual tone	Good rewrite. Reasoning returned separately.
3	German, effort high: word problem with arithmetic	Correct result (9.60 €). Reasoning returned separately.
4	English, effort none: two-sentence summary	Correct. No reasoning content.
5	Tool call get_weather(city)	finish_reason = tool_calls, structured call {"city": "Hamburg"}
6	Long context: 20,060 prompt tokens with an inserted detail	Exact detail retrieved

Perplexity (llama-perplexity -c 512, Kafka from Project Gutenberg with header and footer removed):

Text	PPL
Die Verwandlung (German, 53 chunks)	43.3574 ± 1.28502
Metamorphosis (English, 52 chunks)	40.7886 ± 1.18932

Absolute values are high because of the short window and the 1915 literary text. They are only meant for comparing quants on the same input.

KL divergence against Hob-forge/Kolibri-1-GGUF Q8_0 (revision 40eaa5cf6cfca3afaf8b8832bac962652edb0bde, loaded with the two --override-kv flags; German text, 53 × 512 tokens):

Metric	Value
Mean KLD	0.103851 ± 0.003126
Median KLD	0.030030
Same top token	85.942 ± 0.299 %
PPL (this file / Hob-forge)	43.3574 / 42.4034

These two files are not expected to match. According to its model card, Hob-forge's GGUF was converted from Aleph Alpha's FP8 checkpoint, while this one comes from the BF16 checkpoint. Apart from the two overridden keys, the metadata of both files is identical. Discussion #3 also reports that Hob-forge files with these overrides give the same logits on mainline as on the patched build. The divergence therefore comes from the different source weights, not from the tokenizer, routing or tensor mapping. No comparison against the unquantized BF16 model was run on this hardware.

Informal German writing test

Informal and self-made. This is not a standardized benchmark. Treat it as a hint, not as evidence.

Setup. 14 German writing tasks:

• making officialese understandable
• rewriting a formal text in a casual tone (du)
• cutting a technical text by 50 %
• proofreading a text with 15 planted errors
• turning bullet points into a formal defect notice
• translating EN → DE with a glossary
• matching the style of a sample paragraph
• reducing anglicisms
• gender-neutral wording
• turning notes into a summary
• writing a LinkedIn post
• three longer rewrites, two of them based on public blog articles

Each task ran with 3 seeds, sampling as above, on llama.cpp b11889 (Vulkan) with -c 32768. Only the final answer was rated, never the reasoning.

Rating. An LLM judge (Claude Opus 5.5) rated each output in two independent blind passes with shuffled labels. It scored five criteria on a 1–5 scale: content fidelity, correctness, naturalness, register and instruction following. Automatic checks for spelling and grammar (LanguageTool), required facts and length limits were added on top.

Comparison models. Gemma 4 31B IT (UD-Q8_K_XL), Qwen3.8-27B (Q8_0, official and one community variant) and Qwen3.8-Flash-Next (4-bit).

Effect of reasoning effort

reasoning_effort	Mean score (1–5)	Median completion tokens	Wall time, 42 runs	Truncated
none	4.07	232	5 min	0
low	4.13	1,150	36 min	2 (budget 16k)
high	4.34	3,841	93 min	0 (budget 30k)

• How clear the differences are: high beat none in 99 % and low in 100 % of bootstrap resamples over the tasks. Between low and none there is no clear difference (67 %).
• Constraints: high followed the checkable constraints best, with 97 % met against 89 % for none. For example, it hit the 50 % length target in the shortening task.
• Weak spots at high:
  • In the glossary translation, hyphens were missing ("CSV Export" instead of "CSV-Export").
  • In proofreading, it fixed all 15 errors but also changed 1.3 correct spots on average (e.g. "am" → "Am").
• Token budget: completion tokens include the reasoning. Set max_tokens generously, at least 16k for low and about 30k for high. Otherwise answers get cut off.
• Against the other models: with high, the LLM judge rated Kolibri-1 level with Gemma 4 31B (4.34 vs. 4.30, not significant) and slightly ahead of the Qwen3.8 models (4.14–4.21).

Human blind check (very small sample)

One native German speaker (the uploader) ranked outputs blind, using seed 1 only.

• Round 1: 6 systems on 6 tasks, 5 of which could be ranked. Kolibri-1 ran with none and low, and these took the last two places (mean ranks 4.4 and 4.6 out of 6). Gemma 4 31B came first (1.8). Agreement with the LLM judge was low (mean Spearman 0.11).
• Round 2: Kolibri-1 high against Gemma 4 31B, pairwise on the same 6 tasks. Gemma was preferred 5 : 1, and two of these five decisions were described as close.
  • In the defect notice, Kolibri swapped who had promised the callback ("… und Ihnen einen Rückruf in Aussicht gestellt"). The LLM judge flagged this error too.
  • The human verdict agreed with the LLM judge on 3 of 6 tasks.

Caveat: one rater, one seed, 5–6 tasks per round. That is anecdotal. Human and LLM judgments differ noticeably, so do not rely on the scores alone.

Takeaway: for German writing, use reasoning_effort: high. With none or low, Kolibri-1 is clearly weaker. Even at high, the human rater preferred Gemma 4 31B in this small check.

Performance

llama-bench -p 512,4096 -n 128 -fa 1 -ngl 999, llama.cpp b11889 (d854d98ba), Vulkan (RADV). AMD Ryzen AI Max+ 395 / Radeon 8060S, Mesa 26.2.5, Linux 7.2.14 (Fedora 44):

Test	t/s
pp512	1210.39 ± 21.84
pp4096	1115.14 ± 16.32
tg128	48.51 ± 0.12

Loading the model with llama-server took 19.2 s and used 78.5 GiB of GPU memory (GTT) at -c 32768 with a q8_0 KV cache. In the 20k-token smoke test, prefill ran at 853 t/s and decode at 41 t/s.

Model at a glance and limitations

These points summarize the original model card. Please read it.

• Mixture-of-experts model with 78.1B total and 3.46B active parameters. It has 50 layers with 384 routed experts plus 1 shared expert per layer and top-6 routing. Attention alternates 4:1 between sliding-window GQA (512 preceding tokens plus the current one) and full GQA.
• Text only, German and English only. The knowledge cutoff is June 18, 2026.
• Explicit reasoning mode and tool calling.
• In the vendor's own benchmarks it is weaker at multi-turn tool calling than some comparable models. For example, it scores 38.4 on BFCL v3 multi-turn.
• For the risks, limitations and responsible-use guidance, see the original card.

License and attribution

• The weights are by Aleph Alpha GmbH and are published under Apache 2.0. Per the original card, that license covers only the weights and configuration files. It does not extend to code, model architecture, parameter settings or training methods.
• The LICENSE file from the source repository is included unchanged. The source repository has no NOTICE file.
• Changes: converted to GGUF and quantized to Q8_0; no fine-tuning, no weight changes beyond quantization.

Deutsche Kurzfassung

Dieses Repo enthält Aleph Alphas Kolibri-1 als GGUF in Q8_0, aufgeteilt in fünf Dateien mit zusammen 83,1 GB. Erstellt wurde es mit dem offiziellen Konverter von llama.cpp (b11889) aus den BF16-Gewichten. Es lädt ab llama.cpp b11709 ohne Patch und ohne --override-kv. Beim Start gibt man die erste Datei an, die übrigen Teile lädt llama.cpp selbst. Der Reasoning-Effort wird über chat_template_kwargs mit reasoning_effort gesteuert (none, low, medium, high); ohne Angabe gilt high. In einem informellen deutschen Schreibtest (14 Aufgaben, Bewertung durch ein LLM) war high klar besser als none und low und lag gleichauf mit Gemma 4 31B. In einem sehr kleinen menschlichen Blindtest (1 Person, 6 Aufgaben) wurde Gemma jedoch 5 : 1 vorgezogen. Getestet wurde auf Strix Halo mit Vulkan: pp512 ca. 1210 t/s, tg ca. 48,5 t/s, Perplexity Deutsch 43,36 und Englisch 40,79 (Kafka, -c 512). Es handelt sich um eine inoffizielle Konvertierung, die nicht von Aleph Alpha stammt. Die Lizenz ist Apache 2.0 für Gewichte und Config, Details stehen in der Original-Model-Card.