llm-jp/FS-8x1.5B
Updated
•
28
Drop-Upcycling
-
-
llm-jp/BTX-8x1.5B
Updated • 18 -
llm-jp/FS-8x3.7B
Updated • 17 -
llm-jp/NU-8x1.5B
Updated • 25 -
llm-jp/RNU-0.5-8x1.5B
Updated • 28 -
llm-jp/BTX-8x152M
Updated • 13 -
llm-jp/NU-8x152M
Updated • 33 -
llm-jp/FS-8x152M
Updated • 22 -
llm-jp/DU-0.5-8x152M
Updated • 34 -
llm-jp/DU-0.5-8x1.5B
Updated • 18 -
llm-jp/DU-0.5-8x3.7B
Updated • 26 -
llm-jp/DU-1.0-8x152M
Updated • 19 -
llm-jp/DU-1.0-8x1.5B
Updated • 36 -
llm-jp/RNU-0.5-8x152M
Updated • 32 • 1 -
llm-jp/Dense-1.5B
Updated • 25 -
llm-jp/Dense-152M
Updated • 24 -
llm-jp/Dense-3.7B
Updated • 24 -
llm-jp/Dense-13B
Updated • 17 -
llm-jp/Dense-btx-code-expert-152M
Updated • 17 • 1 -
llm-jp/Dense-btx-english-expert-1.5B
Updated • 19 -
llm-jp/Dense-btx-code-expert-1.5B
Updated • 17 • 1 -
llm-jp/Dense-btx-japanese-expert-1.5B
Updated • 27 • 1 -
llm-jp/Dense-btx-english-expert-152M
Updated • 18 -
llm-jp/Dense-btx-japanese-expert-152M
Updated • 16 -
Drop-Upcycling: Training Sparse Mixture of Experts with Partial Re-initialization
Paper • 2502.19261 • Published • 6