RachidAR (RachidAR)

upvoted a collection 10 days ago

MobileLLM

Optimizing Sub-billion Parameter Language Models for On-Device Use Cases (ICML 2024) https://arxiv.org/abs/2402.14905 • 8 items • Updated 3 days ago • 89

upvoted 2 papers about 1 month ago

Differential Transformer

Paper • 2410.05258 • Published Oct 7 • 165

Addition is All You Need for Energy-efficient Language Models

Paper • 2410.00907 • Published Oct 1 • 143

upvoted 2 collections about 2 months ago

Moshi v0.1 Release

Collection

MLX, Candle & PyTorch model checkpoints released as part of the Moshi release from Kyutai. Run inference via: https://github.com/kyutai-labs/moshi • 13 items • Updated Sep 18 • 215

Qwen2.5

Collection

Qwen2.5 language models, including pretrained and instruction-tuned models of 7 sizes, including 0.5B, 1.5B, 3B, 7B, 14B, 32B, and 72B. • 45 items • Updated Sep 18 • 307

upvoted a paper 2 months ago

OLMoE: Open Mixture-of-Experts Language Models

Paper • 2409.02060 • Published Sep 3 • 77

upvoted 2 papers 5 months ago

Scaling Rectified Flow Transformers for High-Resolution Image Synthesis

Paper • 2403.03206 • Published Mar 5 • 57

Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality

Paper • 2405.21060 • Published May 31 • 63

upvoted 3 papers 6 months ago

upvoted a collection 6 months ago

Papers

Collection

18 items • Updated Oct 7 • 1

upvoted an article 7 months ago

Article

LLM Comparison/Test: Llama 3 Instruct 70B + 8B HF/GGUF/EXL2 (20 versions tested and compared!)

By

•

Apr 24

• 58

upvoted a paper 7 months ago

Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone

Paper • 2404.14219 • Published Apr 22 • 251

upvoted 2 collections 7 months ago

Phi-3

Collection

Phi-3 family of small language and multi-modal models. Language models are available in short- and long-context lengths. • 27 items • Updated 11 days ago • 489

Meta Llama 3

Collection

This collection hosts the transformers and original repos of the Meta Llama 3 and Llama Guard 2 releases • 5 items • Updated Sep 25 • 682

upvoted 4 papers 7 months ago

Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length

Paper • 2404.08801 • Published Apr 12 • 63

Learn Your Reference Model for Real Good Alignment

Paper • 2404.09656 • Published Apr 15 • 82

Rho-1: Not All Tokens Are What You Need

Paper • 2404.07965 • Published Apr 11 • 84

InfLLM: Unveiling the Intrinsic Capacity of LLMs for Understanding Extremely Long Sequences with Training-Free Memory

Paper • 2402.04617 • Published Feb 7 • 4

RachidAR

AI & ML interests

Organizations

RachidAR's activity

MobileLLM

Differential Transformer

Addition is All You Need for Energy-efficient Language Models

Moshi v0.1 Release

Qwen2.5

OLMoE: Open Mixture-of-Experts Language Models

Scaling Rectified Flow Transformers for High-Resolution Image Synthesis

Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality

You Only Cache Once: Decoder-Decoder Architectures for Language Models

Reducing Transformer Key-Value Cache Size with Cross-Layer Attention

TerDiT: Ternary Diffusion Models with Transformers

Papers

LLM Comparison/Test: Llama 3 Instruct 70B + 8B HF/GGUF/EXL2 (20 versions tested and compared!)

Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone

Phi-3

Meta Llama 3

Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length

Learn Your Reference Model for Real Good Alignment

Rho-1: Not All Tokens Are What You Need

InfLLM: Unveiling the Intrinsic Capacity of LLMs for Understanding Extremely Long Sequences with Training-Free Memory