Post
1582
Your RL environment is an SFT data factory 🏭
In LLM post-training it's common to do Supervised Fine-Tuning warm-up before Reinforcement Learning.
When teaching a new task, RL needs some signal to amplify and SFT builds a good initial basis, for example by teaching format.
If you've built an RL env, generating SFT synthetic data is basically free.
An env already has: task data, rollout logic, rewards.
1️⃣ pick a strong model
2️⃣ run it through the env
3️⃣ filter rollouts by reward
works out of the box with Verifiers (Prime Intellect) and Atropos (Nous Research)
🧑💻 Example: https://github.com/anakin87/llm-rl-environments-lil-course/blob/main/chapters/05.md
In LLM post-training it's common to do Supervised Fine-Tuning warm-up before Reinforcement Learning.
When teaching a new task, RL needs some signal to amplify and SFT builds a good initial basis, for example by teaching format.
If you've built an RL env, generating SFT synthetic data is basically free.
An env already has: task data, rollout logic, rewards.
1️⃣ pick a strong model
2️⃣ run it through the env
3️⃣ filter rollouts by reward
works out of the box with Verifiers (Prime Intellect) and Atropos (Nous Research)
🧑💻 Example: https://github.com/anakin87/llm-rl-environments-lil-course/blob/main/chapters/05.md