A 165M model that out-remembers SmolLM2-360M: notes, not scale

Small chat models forget. Ask a 135M or 360M model your name forty exchanges after you said it, and it usually can't tell you. Its attention over a long history is weak, and every new exchange dilutes the few tokens that matter. I wanted to know how small a model can be and still hold a useful multi-turn conversation: remember facts, keep standing instructions, notice false premises. The budget was one Colab TPU chip.

The answer turned out to be: don't make the model remember. Make it take notes, and train it to use them.

On held-out conversations of about 47 exchanges, a 165M model with its own notes scores 0.81 on core memory checks. SmolLM2-360M-Instruct, trained on 4T tokens, scores 0.31 reading the whole conversation in its 8k context. Given the same notes it still scores 0.31, and given perfect notes 0.42.

The resident agent

Each user turn runs in two phases.

1. Note-taking. A note-taker (S6) reads the user's message and writes notes such as rule: call the user Captain or fact: the user lives in Porto, or none. It is sampled three times.
2. Answering. The answerer sees a system prompt, the notes and only the last two exchanges. Nothing older ever enters its context, so a 1024-token window is enough for conversations of any length.

Two details turned out to matter more than expected.

• Grounding with structure, not keywords. Early versions let notes through when the message used the same words. "How can I remember names better?" then became call the user names and silently overwrote "Captain". The fix was structural:

  • questions never set rules;
  • a title must appear capitalised in the message;
  • an ending word must be quoted and present.

  Keyword cue lists looked fine on the development set and failed on new phrasings.

• Two checkpoints, two jobs. RL made the answers better and the notes worse. So the final system uses the pre-RL model (S6) as note-taker and an RL-tuned model as answerer. This needs no extra training, only a second 165M model in memory.

The control that changes the story

The obvious objection: maybe any model does well if you hand it notes. So I ran SmolLM2-360M inside the same agent:

• the same notes from S6;
• the same last-two-exchanges window;
• the same conversations, token for token;
• the same decoding. The repetition penalty applies to generated tokens only; Hugging Face's default also penalises the prompt, which works against copying from notes.

With notes, SmolLM2 recalls entities (0.00 → 0.67) but ignores rule notes (persona 0.08) and does not use them to deny false premises (0.17). Pico was fine-tuned on the note protocol, and RL rewarded using it. The memory is in the training, not just the notes.

On the broader e3 multi-turn suite (160 probes, held-out phrasings), Pico scores 0.68 [0.60, 0.74] against SmolLM2-360M's 0.51 [0.43, 0.58]. Instruction formats are level (0.54 vs 0.54).

What RL with checkable rewards taught, and what it didn't

Every behaviour here has a programmatic check, so I trained three rounds of GRPO with rewards that are just programs, in the spirit of DR Tulu's rubric rewards but with no LLM judge:

• word caps, JSON keys, one sentence;
• an ending word, a title in the first four words;
• a recalled fact, a denial that names the noted fact.

Each round took about an hour on the TPU.

• It reliably taught what is easy to check. Word caps went 0.10 → 1.00 and JSON 0.35 → 0.95. Ending words drawn from a vocabulary that excluded every test word went 0.45 → 0.85 on the test words. Copying generalises.
• It did not teach reading. The identity task used training phrasings. On test phrasings ("And who exactly am I talking to?") the model got worse.
• It games presence checks. Recall rewards check that the fact appears in the reply. The model learned to start everything with "You told me …", including "You told me you're Benedikt." in answer to "who are you?".
• It narrows what it doesn't reward. Answers shrank from 55 to 13 words over two rounds. Most checks are easier with a short reply, and each round's KL anchor is the previous round.
• The first round found a loophole. A canned refusal ("I'm a small AI language model and don't have the ability…") passed several checks at once. A negative rubric (refusals score zero) fixed it.

Weight averaging (WiSE-FT, model soups) helped. Half the pre-RL model plus half the last RL model kept the memory gains (e3 multi-turn 0.68, ending words 0.90), brought answers back to 33 words, and gave the best who-is-who scores. It did not remove the "You told me" habit.

Limitations

This is a research model, not an assistant.

• Knowledge is weak. It invents books, gives poor practical advice, and scores near zero on translation and negation.
• It misreads new phrasings ("What should you call me?" → "You told me your name is Marta").
• It invents things never mentioned ("How is my dog doing?" → "You said you have a dog called Marta").
• The note-taker makes errors: it files pets as the user's name and invents titles from capitalised words.
• The evaluation is limited: one run per configuration, programmatic checks only, and the e3 set guided model selection. The h2 set is the clean test.

Try it

pip install torch tokenizers safetensors huggingface_hub numpy
python pico.py fausap/pico-165m

The repo contains both checkpoints, a small PyTorch port (pico.py) and the JAX notebook that reproduces everything on one Colab TPU chip:

• pretraining, SFT data generation and GRPO;
• every probe set and reward.

Next on my list:

• fine-tune SmolLM2-360M on the same note-protocol data, to separate the recipe from the base model;
• repeat the key comparisons over three seeds.

References

• Allal et al., SmolLM2: When Smol Goes Big, 2025.
• Packer et al., MemGPT: Towards LLMs as Operating Systems, 2023.
• Shao et al., DeepSeekMath (GRPO), 2024.
• Shao et al., DR Tulu, 2025.
• Wortsman et al., Robust fine-tuning of zero-shot models and Model soups, 2022.
• Penedo et al., The FineWeb Datasets, 2024.