Drop-Upcycling
updated
9B • Updated • 108
9B • Updated • 111
19B • Updated • 107
9B • Updated • 110
9B • Updated • 107
0.4B • Updated • 109
0.4B • Updated • 110
0.4B • Updated • 115
0.4B • Updated • 118
9B • Updated • 101
19B • Updated • 114
0.4B • Updated • 112
9B • Updated • 100
0.4B • Updated • 113
• 1
2B • Updated • 119
0.2B • Updated • 129
4B • Updated • 102
14B • Updated • 124
llm-jp/Dense-btx-code-expert-152M
0.2B • Updated • 113
• 1
llm-jp/Dense-btx-english-expert-1.5B
2B • Updated • 107
llm-jp/Dense-btx-code-expert-1.5B
2B • Updated • 112
• 1
llm-jp/Dense-btx-japanese-expert-1.5B
2B • Updated • 210
• 1
llm-jp/Dense-btx-english-expert-152M
0.2B • Updated • 111
llm-jp/Dense-btx-japanese-expert-152M
0.2B • Updated • 155
Drop-Upcycling: Training Sparse Mixture of Experts with Partial
Re-initialization
Paper
• 2502.19261
• Published • 6
Text Generation
• 73B • Updated • 130
llm-jp/llm-jp-3-8x13b-instruct3
Text Generation
• 73B • Updated • 301
• 8
llm-jp/llm-jp-3-8x1.8b-instruct3
Text Generation
• 9B • Updated • 132
• 4
Text Generation
• 9B • Updated • 294
llm-jp/llm-jp-3-8x13b-instruct2
Text Generation
• 73B • Updated • 272
llm-jp/llm-jp-3-8x1.8b-instruct2
Text Generation
• 9B • Updated • 125
llm-jp/llm-jp-3.1-8x13b-instruct4
Text Generation
• 73B • Updated • 231
• 4
Text Generation
• 73B • Updated • 134