ji-farthing/RalphQwen3.8-Flash-Next-GGUF

RalphQwen3.8 Flash Next (tiny Qwen3.8-Flash-Next test fixture)

A 193M-parameter model with the Qwen3.8-Flash-Next architecture. Its weights were randomly initialized, then trained from scratch on a small synthetic corpus. It exists so that Qwen3.8-Flash-Next support can be built and tested without the full model.

This is not a usable language model. It is four layers deep and 320 wide, which is enough to carry the structures of the text model and nothing more. It answers everything with a confident, childlike non-fact, which is what it was trained to do:

I think raindrops put the moon to bed. They sound like the last Tuesday.

Turtles keep tiny houses in their pajamas. The breakfast spoon made me the toast captain!

For use as a fixture, not as a model. It is the Qwen sibling of RalphSeek V4 Flash and RalphSeek V4.1 Flash.

Structure

property	this model	Qwen3.8-Flash-Next
architecture	qwen4exp	same
layers	4: 3 linear attention, then 1 full attention	48: 36 linear attention, 12 full attention
hidden size	320	2560
attention heads / KV heads / head dim	4 / 1 / 128	24 / 2 / 256
linear attention key / value heads	2 / 4	16 / 48
routed experts / used	8 / 2	512 / 10
expert and shared expert width	640	same
hyper-connection streams	4, low rank 320	same
sparse attention indexer	2 heads, compress ratio 4, budget 16	4 heads, compress ratio 4, top-k 2048
per-layer token embedding	on blk.1, n-gram size 3, 2 hashed heads of 4,099 and 4,111 rows, 160 wide	on blk.1, n-gram size 3, 16 hashed heads of about 20 M rows, 160 wide
per-layer token embedding parameters	1.33 M	51.2 G
vocabulary	248,320	same
configured context	4,096	262,144
MTP layers	0	1
vision tower	none	present
parameters	193.02 M

It trained on the RalphSeek corpus of 6,560 training, 816 validation, and 816 test examples. Prompts used the Qwen chat template with thinking disabled, and the loss covered the reply only. Training ran 664 steps, then 2,142 more over two epochs. Final validation loss was 0.441, and test loss was 0.442.

Files

file	size	use
RalphQwen3.8-Flash-Next-F32.gguf	783 MB	every tensor F32

Usage

llama-cli -m RalphQwen3.8-Flash-Next-F32.gguf -c 1024 -fa on --temp 0 -n 64 -e \
          -p "<|im_start|>user\nTeacher: Tell me about raindrops.\nRalph:<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n"

In the corpus a teacher asks and Ralph answers, so a prompt in that shape gets the kind of answer it was trained on.

Other runtimes

The GGUF was converted with the qwen4exp converter in ggml-org/llama.cpp at f8dbcd6. Loading it in upstream llama.cpp is untested. There is no MTP head.

Provenance and licensing

• Weights are original, trained from random initialization on the synthetic corpus used for RalphSeek V4 Flash. They are not derived from, distilled from, or a slice of any Qwen checkpoint. The hidden size alone (320 against 2560) makes that impossible.
• Architecture is the Transformers Qwen4Exp text model (qwen4_exp_text, Transformers 5.18.0.dev0), with the geometry above scaled down from the config.json of Qwen/Qwen3.8-Flash-Next.
• Tokenizer and chat template are copied from the same repository, so that prompts and special tokens behave as they do on the real model. They are the reason for the license above.
• Not affiliated with or endorsed by Qwen or Alibaba.

The name is the same joke as RalphSeek's, about a small thing that says wrong facts with total confidence.