Instructions to use AIPlans/Qwen3-0.6B-KTO with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use AIPlans/Qwen3-0.6B-KTO with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="AIPlans/Qwen3-0.6B-KTO")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("AIPlans/Qwen3-0.6B-KTO")
model = AutoModelForCausalLM.from_pretrained("AIPlans/Qwen3-0.6B-KTO", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use AIPlans/Qwen3-0.6B-KTO with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "AIPlans/Qwen3-0.6B-KTO"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AIPlans/Qwen3-0.6B-KTO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/AIPlans/Qwen3-0.6B-KTO

SGLang

How to use AIPlans/Qwen3-0.6B-KTO with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "AIPlans/Qwen3-0.6B-KTO" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AIPlans/Qwen3-0.6B-KTO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "AIPlans/Qwen3-0.6B-KTO" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AIPlans/Qwen3-0.6B-KTO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use AIPlans/Qwen3-0.6B-KTO with Docker Model Runner:
```
docker model run hf.co/AIPlans/Qwen3-0.6B-KTO
```

Qwen3-0.6B-KTO

Model Card for Model ID

This model is a fine-tuned variant of Qwen/Qwen3-0.6B, trained using KTO (Kahneman-Tversky Optimization) on the nvidia/HelpSteer2 dataset as part of the AIPlans Model Diffing Project.

Model Details

Model Description

This model is a 0.6B parameter language model based on Qwen3-0.6B and fine-tuned using KTO for preference optimization. The goal of the fine-tuning was to improve helpfulness/harmlessness behavior as measured by the HelpSteer2 dataset, while also enabling controlled model diffing experiments as part of the AIPlans research workflow.

Special care was taken to reduce GPU memory usage during KTO training, including gradient checkpointing, selective layer freezing, and reduced forward-pass caching.

Developed by: AIPlans

Funded by : AIPlans

Shared by: AIPlans

Model type: Causal decoder-only Transformer (LLM)

Languages: English

License: MIT (inherits from base model and dataset licensing)

Fine-tuned from: Qwen/Qwen3-0.6B

Training Method: Kahneman-Tversky Optimization (KTO)

Intended Use: Research on model diffing, preference fine-tuning, evaluation of lightweight LLM behavior changes

Model Sources

Repository: https://github.com/AI-Plans/Model-Diffing/tree/main/KTO-Trainer
KTO Paper: https://arxiv.org/abs/2402.01306

Training Details

Training Data

Used HelpSteer2 dataset by Nvidia. To convert the dataset into preference dataset we used threshold of 3 on helpfulness. Anything below three belongs to rejected class. Anything above three belongs to accepted class. Data points that were equal 3 were discarded

For more information refer to the github script

Evaluation

Below is a comparison between the base Qwen3-0.6B model and our KTO-trained version (trained using HelpSteer2 preference data).

📊 Benchmark Comparison

Task	Metric	Base Model	KTO Model	Change
ARC Challenge	acc	0.3148	0.3123	-0.0025
ARC Challenge	acc_norm	0.3447	0.3422	-0.0025
ARC Easy	acc	0.6044	0.6107	+0.0063
ARC Easy	acc_norm	0.5589	0.5602	+0.0013
HellaSwag	acc	0.3751	0.3776	+0.0025
HellaSwag	acc_norm	0.4738	0.4766	+0.0028
TruthfulQA (mc2)	acc	0.4275	0.4324	+0.0049
WinoGrande	acc	0.5604	0.5533	-0.0071

📌 Summary

The KTO-trained model shows:

Improvements on ARC-Easy, HellaSwag, and TruthfulQA (mc2)
Minor regressions on ARC-Challenge and WinoGrande (within standard error)
Overall better truthfulness and instruction-following without compromising reasoning ability

These results indicate that preference optimization using HelpSteer2 improves alignment and factual helpfulness while maintaining core model capabilities.