Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of-Expert Models
Paper
•
2501.11873
•
Published
•
59
Yixin Song
yixinsong
AI & ML interests
None yet
Recent Activity
updated
a model
10 days ago
PowerInfer/SmallThinker-3B-Preview
new activity
10 days ago
PowerInfer/SmallThinker-3B-Preview:Eval script
Organizations
yixinsong's activity
upvoted
a
paper
4 days ago
updated
a
model
10 days ago
Eval script
4
#9 opened 10 days ago
by
rawsh
liked
a
dataset
18 days ago
About the training details
5
#5 opened 22 days ago
by
hiyouga
How to Pair with Larger Models
4
#7 opened 21 days ago
by
windkkk
Prompt/token adjust to stop "Overthinking" in unnescissary cases
2
#6 opened 22 days ago
by
fuzzy-mittenz
example use colab?
8
#3 opened 23 days ago
by
NickyNicky
Update README.md
#4 opened 23 days ago
by
AISafety
liked
a
dataset
23 days ago
Training: Second Phase
6
#2 opened 26 days ago
by
tugstugi
[bot] Conversion to Parquet
#1 opened about 2 months ago
by
parquet-converter
[bot] Conversion to Parquet
#1 opened 25 days ago
by
parquet-converter
Librarian Bot: Add language metadata for dataset
#2 opened 25 days ago
by
librarian-bot
liked
a
model
24 days ago
Training: Second Phase
6
#2 opened 26 days ago
by
tugstugi
updated
a
model
26 days ago
Evaluation
6
#1 opened 27 days ago
by
tugstugi
updated
a
dataset
26 days ago
updated
a
model
26 days ago