Instructions to use LatitudeGames/Muse-12B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use LatitudeGames/Muse-12B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="LatitudeGames/Muse-12B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("LatitudeGames/Muse-12B")
model = AutoModelForCausalLM.from_pretrained("LatitudeGames/Muse-12B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use LatitudeGames/Muse-12B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "LatitudeGames/Muse-12B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LatitudeGames/Muse-12B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/LatitudeGames/Muse-12B

SGLang

How to use LatitudeGames/Muse-12B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "LatitudeGames/Muse-12B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LatitudeGames/Muse-12B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "LatitudeGames/Muse-12B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LatitudeGames/Muse-12B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use LatitudeGames/Muse-12B with Docker Model Runner:
```
docker model run hf.co/LatitudeGames/Muse-12B
```

Excellent Model

by Utochi - opened Jul 11, 2025

Discussion

Utochi

Jul 11, 2025

I would humbly like to say that this is the best 12B model that i know of.

Using the recommended settings, its been a great companion for me for quite some time now, its intuitive and easy to work with. While i am experimenting with larger models, this is still such a nice model to have in ones arsenal, ive only ever used it on q8, temp 0.8.

It can be a BIT chatty, Usually i have in my system prompt something to tell it to tone it down a bit.

I have had zero refusals with this model so far. it can be a bit horny though, not overly so.. it just really wants to take a situation that should bring hatred or fear and add in a horny aspect to it like "She's utterly terrified... but a traitorous part of her cant help--" etc so that was one thing i always found a little annoying about it but never enough to be a bother.

Currently the only model that I've seen that really beats this one is Cydonia v4 from my current testing, which id like to mention that even the other versions of Cydonia in my opinion does not match this one as far as just.. comfort goes. when i find a model i really like i stick with it and lightly test other ones, it has to impress me, reasonably work with me. sure ill match the recommended settings but the model needs to adapt to my preferences and style, that makes for a good round about model and this one was that one.

my roleplay style is; Action "Dialogue" and rarely Thought

Skyletwings

13 days ago

•

edited 13 days ago

That sounds great!

But I'm a noob, trying to call KoboldCPP having this model loaded. The POST endpoint "generate" accepts a prompt, but I do not know the format for presenting my system-prompt, author's notes, memory, etc.
Would you please provide me (and other help seeking fans) a shape/template to format my prompt to KoboldCPP for usage with this model - or hint to any source that describes it? The model description meantions that ChatML was used for training, does that mean I have to format the requests according to standard ChatML format?

Thank you very much!

Utochi

12 days ago

@Skyletwings truth be told its been quite a while since I've used this model and i dont use KoboldCPP BUT, yes I do remember that ChatML standard worked fine with it. My current recommendation though as a change would be either MagMel R1 Q6 (ChatML) OR G4 MeroMero Q6 (Gemma4), i dont remember why i switched away from Muse, but im sure i had a reason xD

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment