Extending the Context of Pretrained LLMs by Dropping Their Positional Embedding (https://www.arxiv.org/abs/2512.12167)
AI & ML interests
We are a Tokyo-based R&D company on a quest to create a new kind of foundational AI model based on nature-inspired intelligence.
Recent Activity
Papers
View all PapersStudents distilled from a 7B Reinforcement-Learned Teacher (RLT) from the paper "Reinforcement Learning Teachers of Test Time Scaling."
-
SakanaAI/Llama-3-8B-Instruct-CycleQD-CS
Text Generation • 8B • Updated • 4 • 1 -
SakanaAI/Llama-3-8B-Instruct-Coding-Expert
Text Generation • 8B • Updated • 20 • • 13 -
SakanaAI/Llama-3-8B-Instruct-DB-Expert
Text Generation • 8B • Updated • 4 -
SakanaAI/Llama-3-8B-Instruct-OS-Expert
Text Generation • 8B • Updated • 7 • 1
because thought takes time and reasoning is a process.
Compact Japanese models trained with "TAID: Temporally Adaptive Interpolated Distillation for Efficient Knowledge Transfer in Language Models"
-
SakanaAI/TinySwallow-1.5B
Text Generation • 2B • Updated • 3.69k • • 35 -
SakanaAI/TinySwallow-1.5B-Instruct
Text Generation • 2B • Updated • 11.8k • • 57 -
SakanaAI/TinySwallow-1.5B-Instruct-q4f32_1-MLC
Text Generation • Updated • 3 -
SakanaAI/TinySwallow-1.5B-Instruct-GGUF
Text Generation • 2B • Updated • 530 • 27
Extending the Context of Pretrained LLMs by Dropping Their Positional Embedding (https://www.arxiv.org/abs/2512.12167)
because thought takes time and reasoning is a process.
Students distilled from a 7B Reinforcement-Learned Teacher (RLT) from the paper "Reinforcement Learning Teachers of Test Time Scaling."
Compact Japanese models trained with "TAID: Temporally Adaptive Interpolated Distillation for Efficient Knowledge Transfer in Language Models"
-
SakanaAI/TinySwallow-1.5B
Text Generation • 2B • Updated • 3.69k • • 35 -
SakanaAI/TinySwallow-1.5B-Instruct
Text Generation • 2B • Updated • 11.8k • • 57 -
SakanaAI/TinySwallow-1.5B-Instruct-q4f32_1-MLC
Text Generation • Updated • 3 -
SakanaAI/TinySwallow-1.5B-Instruct-GGUF
Text Generation • 2B • Updated • 530 • 27
-
SakanaAI/Llama-3-8B-Instruct-CycleQD-CS
Text Generation • 8B • Updated • 4 • 1 -
SakanaAI/Llama-3-8B-Instruct-Coding-Expert
Text Generation • 8B • Updated • 20 • • 13 -
SakanaAI/Llama-3-8B-Instruct-DB-Expert
Text Generation • 8B • Updated • 4 -
SakanaAI/Llama-3-8B-Instruct-OS-Expert
Text Generation • 8B • Updated • 7 • 1