llm-jp/FS-8x1.5B
Updated
•
8
Drop-Upcycling
-
-
llm-jp/BTX-8x1.5B
Updated • 4 -
llm-jp/FS-8x3.7B
Updated • 4 -
llm-jp/NU-8x1.5B
Updated • 7 -
llm-jp/RNU-0.5-8x1.5B
Updated • 5 -
llm-jp/BTX-8x152M
Updated • 4 -
llm-jp/NU-8x152M
Updated • 11 -
llm-jp/FS-8x152M
Updated • 7 -
llm-jp/DU-0.5-8x152M
Updated • 8 -
llm-jp/DU-0.5-8x1.5B
Updated • 5 -
llm-jp/DU-0.5-8x3.7B
Updated • 4 -
llm-jp/DU-1.0-8x152M
Updated • 4 -
llm-jp/DU-1.0-8x1.5B
Updated • 4 -
llm-jp/RNU-0.5-8x152M
Updated • 5 • 1 -
llm-jp/Dense-1.5B
Updated • 5 -
llm-jp/Dense-152M
Updated • 11 -
llm-jp/Dense-3.7B
Updated • 9 -
llm-jp/Dense-13B
Updated • 2 -
llm-jp/Dense-btx-code-expert-152M
Updated • 4 • 1 -
llm-jp/Dense-btx-english-expert-1.5B
Updated • 3 -
llm-jp/Dense-btx-code-expert-1.5B
Updated • 2 • 1 -
llm-jp/Dense-btx-japanese-expert-1.5B
Updated • 4 • 1 -
llm-jp/Dense-btx-english-expert-152M
Updated • 3 -
llm-jp/Dense-btx-japanese-expert-152M
Updated • 4 -
Drop-Upcycling: Training Sparse Mixture of Experts with Partial Re-initialization
Paper • 2502.19261 • Published • 4