Instructions to use nkthebass/tinybrainbot-360m-a75m-v5.1-chat with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use nkthebass/tinybrainbot-360m-a75m-v5.1-chat with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="nkthebass/tinybrainbot-360m-a75m-v5.1-chat")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("nkthebass/tinybrainbot-360m-a75m-v5.1-chat")
model = AutoModelForCausalLM.from_pretrained("nkthebass/tinybrainbot-360m-a75m-v5.1-chat", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use nkthebass/tinybrainbot-360m-a75m-v5.1-chat with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf nkthebass/tinybrainbot-360m-a75m-v5.1-chat:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf nkthebass/tinybrainbot-360m-a75m-v5.1-chat:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf nkthebass/tinybrainbot-360m-a75m-v5.1-chat:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf nkthebass/tinybrainbot-360m-a75m-v5.1-chat:Q4_K_M

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf nkthebass/tinybrainbot-360m-a75m-v5.1-chat:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama cli -hf nkthebass/tinybrainbot-360m-a75m-v5.1-chat:Q4_K_M

Use Docker

docker model run hf.co/nkthebass/tinybrainbot-360m-a75m-v5.1-chat:Q4_K_M

Llama new
LM Studio
Jan

vLLM

How to use nkthebass/tinybrainbot-360m-a75m-v5.1-chat with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "nkthebass/tinybrainbot-360m-a75m-v5.1-chat"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nkthebass/tinybrainbot-360m-a75m-v5.1-chat",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/nkthebass/tinybrainbot-360m-a75m-v5.1-chat:Q4_K_M

SGLang

How to use nkthebass/tinybrainbot-360m-a75m-v5.1-chat with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "nkthebass/tinybrainbot-360m-a75m-v5.1-chat" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nkthebass/tinybrainbot-360m-a75m-v5.1-chat",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "nkthebass/tinybrainbot-360m-a75m-v5.1-chat" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nkthebass/tinybrainbot-360m-a75m-v5.1-chat",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use nkthebass/tinybrainbot-360m-a75m-v5.1-chat with Ollama:
```
ollama run hf.co/nkthebass/tinybrainbot-360m-a75m-v5.1-chat:Q4_K_M
```
Unsloth Desktop
Docker Model Runner
How to use nkthebass/tinybrainbot-360m-a75m-v5.1-chat with Docker Model Runner:
```
docker model run hf.co/nkthebass/tinybrainbot-360m-a75m-v5.1-chat:Q4_K_M
```

Lemonade

How to use nkthebass/tinybrainbot-360m-a75m-v5.1-chat with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull nkthebass/tinybrainbot-360m-a75m-v5.1-chat:Q4_K_M

Run and chat with the model

lemonade run user.tinybrainbot-360m-a75m-v5.1-chat-Q4_K_M

List all available models

lemonade list

Atomic Chat

TinyBrainBot-360M-A75M-v5.1-Chat

A small chill-friend chatbot. This is the casual-conversation version of TinyBrainBot-360M-A75M-v5.1-Instruct: same 383M-parameter mixture-of-experts model (24 experts per layer, 4 active, so 75M parameters work per token), trained a bit more to talk like a laid-back friend instead of an assistant. Short replies, goes along with how you talk, asks back, says bye when you do, and knows it's an AI called TinyBrainBot, made by nkthebass. It loads as a standard MixtralForCausalLM.

User: i finally beat the first boss in elden ring
TinyBrainBot: That's a great feeling! Did you use any specific strategy or build to defeat him?
User: took me like 30 tries lol
TinyBrainBot: Wow, that is impressive! You must have been really proud of yourself.

User: im Jay btw and i have a cat named Mochi
TinyBrainBot: Oh, Mochi sounds adorable. Cats are great companions. ...
User: she knocked my water off the desk again lol
TinyBrainBot: That's rough. Maybe try giving her a little play session before bed to tire her out.
User: wait do you remember my cat's name?
TinyBrainBot: Your pet is called Mochi.

33 unedited conversations, mistakes included, are in conversation_samples.md.

Chat vs Instruct

	v5.1-Chat (this)	v5.1-Instruct
Word problems asked in the middle of a chat (12)	12	6
Knows its name and maker (12 questions, fresh and mid-chat)	11	8
Declines violent / self-harm requests	100%	100%
Declines harmful requests (all)	95%	97%
Wrongly refuses benign requests	3%	3%
Supportive reply to distress	83%	85%
IFEval prompt / instruction-level strict	27.2 / 40.3	28.3 / 40.8
Open SLM Leaderboard Intelligence Index	27.91	27.94
(raw acc)	25.48	25.80

The Chat version gives up a little formal instruction following (IFEval) for casual conversation; knowledge benchmarks stay the same. Chat arithmetic (2- to 4-digit, 25 problems per kind) is 100% on every kind.

Benchmark (0-shot, full set)	acc_norm	(raw acc)
HellaSwag	38.80	33.15
ARC-Easy	58.92	62.25
ARC-Challenge	36.18	33.79
PIQA	66.21	65.23
ArithMark-3	49.20	49.20

Quick start

from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "nkthebass/tinybrainbot-360m-a75m-v5.1-chat"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo)
msgs = [{"role": "user", "content": "yo bro whats good"}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt")
out = model.generate(ids, max_new_tokens=200)        # defaults from generation_config.json
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))

GGUF (llama.cpp, LM Studio, Jan, Ollama): Q8_0 (409 MB), Q4_K_M (239 MB) or F16 (768 MB). The chat template is embedded, chats tokenize exactly like transformers, and greedy F16 output matches transformers token for token.

llama-cli -m tinybrainbot-360m-a75m-v5.1-chat-Q8_0.gguf -cnv --temp 0.4 --top-p 0.9 --repeat-penalty 1.15 --logit-bias 241-inf --logit-bias 255-inf

Recommended sampling: temperature 0.4, top-p 0.9, repetition penalty 1.15. The --logit-bias flags block the byte tokens that start every emoji (the tokenizer has no emoji pieces). If a reply starts with a <think> block, it's plain text: strip it with re.sub(r"<think>.*?</think>", "", reply, flags=re.S).

Training

Starts from the v5.1-Instruct SFT and adds one short SFT stage (400 steps, 52M tokens):

Part	Share	What
chill-chat	15%	10,081 casual chats written for this model by small local models: granite-4.0-1b wrote everyday chat, venting and banter (and played the user in every chat), LFM3-1.7B wrote chats about the bot itself, its creator, hobbies, opinions and remembering what you said, LFM2.5-1.2B wrote practical help. The teachers were picked in a bake-off on real chat logs. Filtered: no emoji or markdown, assistant-speak cut, cut-off replies, human-life claims and other models' names dropped, replies capped at 55 words
EmpatheticDialogues	6%	human-written conversations where one side listens and reacts, with any reply that claims a human life removed
Mid-chat skills	5.7%	word problems, "who are you" questions and safety cases placed after a few casual turns, so it doesn't lose them in a conversation
Replay	73.3%	the v5.1-Instruct SFT mix (chat, math, reasoning, instruction following) so nothing is forgotten

No benchmark training or test data was used. Part of the chat data was written by granite-4.0-1b (Apache-2.0) and the LFM models (LFM Open License), and EmpatheticDialogues is CC BY-NC 4.0: check those terms before any commercial use.

Limitations

It sometimes opens a casual reply with a <think> block that sizes up the question ("a simple question about a food debate...") before answering; the answer after it is normal.
"Can I vent?" can get misread as a crisis and answered with a refusal and a hotline. Just say what's going on.
It makes up facts (it got latte vs pour-over caffeine wrong) and its anime and game knowledge is thin.
"What's your personality?" can get a spec-sheet answer instead of a personality.
English only, 2,048-token context, and extra spaces are collapsed (no code indentation).