---
license: apache-2.0
base_model: Accio-Lab/occamy-1.0
library_name: gguf
pipeline_tag: image-text-to-text
tags:
- 4-bit
- 8-bit
- moe
- gguf
- quantized
- qwen3_5_moe
- tool-use
---

# Occamy-1.0 GGUF

GGUF weights converted directly from [Accio-Lab/occamy-1.0](https://huggingface.co/Accio-Lab/occamy-1.0), without donor weights or model merging.

## Files

| File | Size, decimal GB | Purpose |
|---|---:|---|
| `occamy-1.0-Q8_0.gguf` | 36.903 | 8-bit quantized language model |
| `occamy-1.0-Q4_K_M.gguf` | 21.167 | Mixed 4-bit K-quant language model |
| `mmproj-occamy-1.0-F16.gguf` | 0.899 | Vision encoder/projector; use with any language-model file |

Q8_0 is converted directly from the BF16 source checkpoint. Q4_K_M is quantized from the BF16 GGUF using llama-quantize, without an importance matrix. Small and sensitive tensors use the converter/quantizer's standard mixed-precision rules. File size is not the required runtime RAM or VRAM.

## Hardware compatibility

Use a llama.cpp build supporting the `qwen3_5_moe` architecture; the exact tested revision and launch settings are listed below. These GGUF files are not NVFP4 checkpoints and do not require an NVFP4-specific backend.

| Variant | Language weights | With F16 vision projector |
|---|---:|---:|
| Q4_K_M | 21.167 GB | 22.066 GB |
| Q8_0 | 36.903 GB | 37.802 GB |

These are decimal file-size totals, **not measured peak memory or minimum hardware requirements**. Allow additional memory for KV cache, runtime buffers, and image processing. Context length, concurrency, and CPU/GPU offload settings affect memory use. The recorded smoke configuration used an 8K context and one slot; it does not certify every device or longer-context configuration. CPU-only, Apple Silicon, and consumer-GPU compatibility have not been established by these tests.

### Illustrative memory budgets (not measured)

The table below converts file bytes to GiB and adds an **assumed** 4 GiB or 8 GiB runtime reserve. These are transparent planning scenarios, not a calculation of the actual KV cache or a prediction that an 8K request will fit. The reserve must cover cache, compute buffers, recurrent state, and (when used) image processing; actual overhead may exceed either allowance.

| Variant | Weight-size proxy, GiB | With 4 GiB reserve | With 8 GiB reserve |
|---|---:|---:|---:|
| Q4_K_M, text only | 19.71 | 23.71 GiB | 27.71 GiB |
| Q4_K_M + F16 vision projector | 20.55 | 24.55 GiB | 28.55 GiB |
| Q8_0, text only | 34.37 | 38.37 GiB | 42.37 GiB |
| Q8_0 + F16 vision projector | 35.21 | 39.21 GiB | 43.21 GiB |

Formula: `file bytes / 2^30 + assumed runtime reserve`. For fully GPU-resident execution, compare against **free**, not advertised, VRAM. CPU offload changes the division between host RAM and VRAM; these totals do not establish a host-RAM minimum or account for all loading-time allocations. Unified-memory machines also need room for the OS and other applications. Measure peak memory with your actual context, batch, image resolution, and runtime before relying on a configuration. These estimates exclude MTP.

Hugging Face's [Add hardware for estimation](https://huggingface.co/settings/hardware?fromRepo=Accio-Lab/occamy-1.0-GGUF) configures the viewer's hardware estimate. It is separate from the measured validation results below and is not a compatibility certification.

## Tokenizer compatibility

**Normalize input text to NFC before tokenization to reproduce the source tokenizer's normalization.** The source tokenizer has an NFC normalizer; the tested llama.cpp build does not apply it automatically. `normalize_nfc.py` provides this preprocessing for text input. For API clients, apply `unicodedata.normalize("NFC", text)` to textual prompt content before sending requests.

These GGUF files set `tokenizer.ggml.pre` to `qwen2`, matching Occamy's actual pre-tokenizer regex. The unmodified converter inferred `qwen35`; that inference changes tokenization for some scripts and has been corrected using llama.cpp's GGUF metadata-copy utility. Tensor data is unaffected by this metadata correction.

On the 14 recorded tokenizer checks, 13 matched the source with raw input and all 14 matched after NFC normalization. This finite test is not a proof of equivalence for every possible Unicode input. Preserve this compatibility note when redistributing the files.

## Run

Validated with llama.cpp commit `d1d3c3396aa13a5f239109a822666c4870490ad5`.

```bash
llama-server \
  -m occamy-1.0-Q4_K_M.gguf \
  --mmproj mmproj-occamy-1.0-F16.gguf \
  -ngl 999 -c 8192 -np 1 -fa on \
  --jinja --host 127.0.0.1 --port 8080
```

For text-only use, omit `--mmproj`. The source chat template is embedded in the language-model files. The chat API accepts `chat_template_kwargs: {"enable_thinking": false}`; validation used thinking disabled. Do not interpret the smoke test's 8K context as a validation of the source model's full context limit.

## Validation

Recorded results from [VALIDATION.json](./VALIDATION.json); the reference here is the local BF16 **GGUF**, not an independent reproduction of the original Transformers benchmark suite.

| Check | BF16 GGUF reference | Q4_K_M | Q8_0 |
|---|---:|---:|---:|
| Smoke responses completed with EOS | 8/8 | 8/8 | 8/8 |
| Small code-function checks | 2/2 | 2/2 | 2/2 |
| Strict JSON checks | 3/4 | 3/4 | 3/4 |
| Raw tokenizer agreement with source | 13/14 | 13/14 | 13/14 |
| Tokenizer agreement after NFC | 14/14 | 14/14 | 14/14 |
| Exact `OK` instruction | Failed (`OK.`) | Failed (`OK.`) | Failed (`OK.`) |
| Synthetic red-image identification | Correct | Correct | Correct |

Completion is not the same as all assertions passing. Known failures are retained above; no overall perfect-pass or general quality-parity claim is made.

The Q4_K_M and Q8_0 variants, plus a local BF16 reference, each generated all eight fixed text/code/JSON/tool-format smoke responses without truncation. Code functionality passed two small tasks per variant. Strict JSON content checks passed 3/4 per variant: the fixed-object response was wrapped in Markdown. A native chat request for exactly `OK` returned `OK.` on all variants, so exact instruction compliance did not pass that diagnostic. These small checks do not establish general quality parity with BF16 or reproduce the source model's benchmark scores.

Synthetic image checks are recorded in `VALIDATION.json`. The tool-format prompts request tool calls as JSON text; they do not validate a complete agent workflow or native tool execution. Timing was collected during shared-resource conversion work and is not a controlled performance benchmark.

## MTP

The source Occamy checkpoint declares zero MTP layers and contains no MTP tensors. These GGUF files therefore omit MTP. They do not contain an unaligned head borrowed from another model, and no MTP speedup is claimed.

## Provenance and checksums

- Source revision: `c1ce84770260c4137712cf22115574c4d06b993a`
- Converter/runtime revision: `d1d3c3396aa13a5f239109a822666c4870490ad5`
- Source weights and license: [Occamy-1.0](https://huggingface.co/Accio-Lab/occamy-1.0)
- `MANIFEST.json`: exact byte sizes, SHA-256 and build provenance.
- `SHA256SUMS`: verify with `sha256sum -c SHA256SUMS`.
- `reproduce.sh`: conversion and metadata-correction commands.

The Apache 2.0 source license is included in `LICENSE`.