Instructions to use ProCreations/grug-27b-v2-gguf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use ProCreations/grug-27b-v2-gguf with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf ProCreations/grug-27b-v2-gguf:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf ProCreations/grug-27b-v2-gguf:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf ProCreations/grug-27b-v2-gguf:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf ProCreations/grug-27b-v2-gguf:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf ProCreations/grug-27b-v2-gguf:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf ProCreations/grug-27b-v2-gguf:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf ProCreations/grug-27b-v2-gguf:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf ProCreations/grug-27b-v2-gguf:Q4_K_M

Use Docker

docker model run hf.co/ProCreations/grug-27b-v2-gguf:Q4_K_M

Llama new
LM Studio
Jan

vLLM

How to use ProCreations/grug-27b-v2-gguf with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ProCreations/grug-27b-v2-gguf"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ProCreations/grug-27b-v2-gguf",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ProCreations/grug-27b-v2-gguf:Q4_K_M

Ollama
How to use ProCreations/grug-27b-v2-gguf with Ollama:
```
ollama run hf.co/ProCreations/grug-27b-v2-gguf:Q4_K_M
```
Unsloth Desktop

How to use ProCreations/grug-27b-v2-gguf with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf ProCreations/grug-27b-v2-gguf:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "ProCreations/grug-27b-v2-gguf:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use ProCreations/grug-27b-v2-gguf with Docker Model Runner:
```
docker model run hf.co/ProCreations/grug-27b-v2-gguf:Q4_K_M
```

Lemonade

How to use ProCreations/grug-27b-v2-gguf with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull ProCreations/grug-27b-v2-gguf:Q4_K_M

Run and chat with the model

lemonade run user.grug-27b-v2-gguf-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use ProCreations/grug-27b-v2-gguf with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf ProCreations/grug-27b-v2-gguf:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default ProCreations/grug-27b-v2-gguf:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use ProCreations/grug-27b-v2-gguf with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf ProCreations/grug-27b-v2-gguf:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "ProCreations/grug-27b-v2-gguf:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

grug 27b v2 — gguf

grug brain squeezed into smaller rock. draft head come with rock now.

these quantizations of ProCreations/grug-27b-v2. every text GGUF carry Grug-tuned native MTP head. no separate draft model to fetch. optional mmproj file is preserved vision projector — eye rock, not draft brain.

pick your rock

File	Size (GiB)	Embedded MTP
mmproj-grug-27b-v2-F16.gguf	0.86	Vision projector
grug-27b-v2-Q3_K_M.gguf	12.81	15 tensors; matrices Q8_0
grug-27b-v2-Q4_K_M.gguf	15.83	15 tensors; matrices Q8_0
grug-27b-v2-Q5_K_M.gguf	18.33	15 tensors; matrices Q8_0
grug-27b-v2-Q6_K.gguf	20.99	15 tensors; matrices Q8_0
grug-27b-v2-Q8_0.gguf	27.05	15 tensors; matrices Q8_0

grug test squeezed brain too

rocks converted and tested on HF Jobs with llama.cpp commit 2a3005c23f60cb38dab70b8ea2ddbd969bcf3e87. every quant load on H200, MTP off and on. grug check generation, titles, tools, history handling, executable code.

results include 16-case coding smoke test for each quant. small smoke test, not full quantized benchmark. grug not claim rock pass hunt it never take.

how wake rock

llama-server \
  -hf ProCreations/grug-27b-v2-gguf:Q4_K_M \
  -ngl 99 -c 12288 --jinja \
  --temp 0.6 --top-p 0.95 --top-k 20 \
  --repeat-penalty 1.05 --repeat-last-n 12288 \
  --spec-type draft-mtp --spec-draft-n-max 2 -ngld 99

pick low, medium, xhigh through chat endpoint's reasoning_effort. default medium. bigger effort give grug more thought, not guaranteed better score. hard thought sometimes leave Grug style. some loops still live. main model card and full results show bruise too.

keep old thought in right pouch

pinned llama-server accept canonical reasoning_content and inline history. it drop reasoning input alias before template render. grug record this as one failed capability probe in each mode, not sweep under cave rug.

forwarding vLLM-style histories? use normalize_messages.py. usage.md explain plumbing.

what rock not promise

MTP head packed by default. speculative decoding still need runtime switch. MTP paths not guaranteed identical greedy tokens, speed gain not universal.

vision weights preserved, but no new vision benchmark. full configured-context quality not established. grug keep eye; grug not invent eye score.

rock receipts

gguf_conversion.json record files and embedded head counts. check downloads against SHA256SUMS. all model compute happen on HF Jobs; HF Jobs ledger cover whole main-model and GGUF build. Apache-2.0 license.

grug made by ProCreations. base brain by Qwen team.

Downloads last month: 539

GGUF

Model size

27B params

Architecture

qwen35

Hardware compatibility

3-bit

4-bit

5-bit

6-bit

8-bit

Model tree for ProCreations/grug-27b-v2-gguf

Base model

Qwen/Qwen3.8-27B

Finetuned

ProCreations/grug-27b-v2

Quantized

(3)

this model

Collection including ProCreations/grug-27b-v2-gguf

Grug Series

Collection

A collection of all Grug models I have made. • 22 items • Updated Sep 25 • 5