acon96/Little-Titles

Little Titles

Little Titles is a fine-tuned Qwen/Qwen3.5-0.8B-Base model that generates a concise title for a user request or chat conversation. The goal is to provide a compact, but accurate, model for generating chat titles.

Usage

Input contract

The saved tokenizer chat template is intentionally restrictive. Each request must contain exactly one user message, optionally preceded by one system message. It does not accept multi-turn conversations, prior assistant messages, or tools. To title a full conversation, serialize or summarize that conversation into the content of the single user message.

Set add_generation_prompt=True when calling the Transformers chat template. Without a system message, the template uses: Generate a short title describing the following user request.

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "acon96/Little-Titles"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")

messages = [{"role": "user", "content": "Help me plan a three-day trip to Kyoto."}]
inputs = tokenizer.apply_chat_template(
        messages, add_generation_prompt=True, return_tensors="pt", return_dict=True
).to(model.device)
output = model.generate(**inputs, max_new_tokens=32, do_sample=False)
title = tokenizer.decode(output[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True)
print(title.strip())

To supply a custom instruction, prepend one system message while retaining one user message:

messages = [
    {"role": "system", "content": "Return a concise, action-oriented title."},
    {"role": "user", "content": "Help me plan a three-day trip to Kyoto."},
]

Deployment with vLLM

Serve the model through an OpenAI-compatible API:

vllm serve acon96/Little-Titles \
    --host 0.0.0.0 \
    --port 8000 \
    --max-model-len 32768

Then request a title through /v1/chat/completions:

curl http://localhost:8000/v1/chat/completions \
    -H 'Content-Type: application/json' \
    -d '{
        "model": "acon96/Little-Titles",
        "messages": [{"role": "user", "content": "Help me plan a three-day trip to Kyoto."}],
        "temperature": 0,
        "max_tokens": 32
    }'

Training

The fine-tuning configuration is available in training-config.yml. It uses SupraLabs/chat-titles-filtered-115K and Axolotl to fine-tune the base model.

Evaluation

We compare Little Titles with a reference title supplied by the ogrnz/chat-titles dataset. Gemma 4 26B A4B rated each pair for accuracy, relevance, and concise usefulness.

Across 10,000 paired judgments, Little Titles won 4,831 comparisons versus 3,531 reference-title wins (1,638 ties). Its decisive-pair win rate was 57.8% (95% CI: 56.7%–58.8%; exact one-sided sign test $p = 2.81 \times 10^{-46}$), so it was preferred more often than the reference title in this evaluation.

This is exploratory evidence: Little Titles was always Candidate A, so candidate-position bias may affect this result.