innovation64 (Yang Lee)

upvoted a paper about 1 month ago

DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models

Paper • 2401.06066 • Published Jan 11 • 42

upvoted a paper about 2 months ago

SAM 2: Segment Anything in Images and Videos

Paper • 2408.00714 • Published Aug 1 • 103

upvoted 2 articles about 2 months ago

Article

makeMoE: Implement a Sparse Mixture of Experts Language Model from Scratch

By

•

May 7

• 36

Article

Sparse Mixture of Experts Language Model from Scratch: Extending makeMoE with Expert Capacity

By

•

Mar 18

• 7

upvoted a paper about 2 months ago

ChatQA 2: Bridging the Gap to Proprietary LLMs in Long Context and RAG Capabilities

Paper • 2407.14482 • Published Jul 19 • 24

upvoted 6 papers 2 months ago

Is Your Model Really A Good Math Reasoner? Evaluating Mathematical Reasoning with Checklist

Paper • 2407.08733 • Published Jul 11 • 20

Self-Recognition in Language Models

Paper • 2407.06946 • Published Jul 9 • 24

upvoted 4 papers 3 months ago

From RAGs to rich parameters: Probing how language models utilize external knowledge over parametric information for factual queries

Paper • 2406.12824 • Published Jun 18 • 20

ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools

Paper • 2406.12793 • Published Jun 18 • 31

Test of Time: A Benchmark for Evaluating LLMs on Temporal Reasoning

Paper • 2406.09170 • Published Jun 13 • 24

Transformers meet Neural Algorithmic Reasoners

Paper • 2406.09308 • Published Jun 13 • 43

upvoted an article 3 months ago

Article

Putting RL back in RLHF

Jun 12

• 58

upvoted 4 papers 3 months ago

CRAG -- Comprehensive RAG Benchmark

Paper • 2406.04744 • Published Jun 7 • 40

WildBench: Benchmarking LLMs with Challenging Tasks from Real Users in the Wild

Paper • 2406.04770 • Published Jun 7 • 26

NATURAL PLAN: Benchmarking LLMs on Natural Language Planning

Paper • 2406.04520 • Published Jun 6 • 10

GenAI Arena: An Open Evaluation Platform for Generative Models

Paper • 2406.04485 • Published Jun 6 • 19

upvoted an article 3 months ago

Article

Making sense of this mess

Jun 7

• 14

upvoted 6 papers 4 months ago

Similarity is Not All You Need: Endowing Retrieval Augmented Generation with Multi Layered Thoughts

Paper • 2405.19893 • Published May 30 • 28

Matryoshka Multimodal Models

Paper • 2405.17430 • Published May 27 • 30

MoRA: High-Rank Updating for Parameter-Efficient Fine-Tuning

Paper • 2405.12130 • Published May 20 • 45

Layer-Condensed KV Cache for Efficient Inference of Large Language Models

Paper • 2405.10637 • Published May 17 • 19

LoRA Learns Less and Forgets Less

Paper • 2405.09673 • Published May 15 • 86

RAFT: Adapting Language Model to Domain Specific RAG

Paper • 2403.10131 • Published Mar 15 • 66

upvoted 3 papers 5 months ago

Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Models

Paper • 2405.01535 • Published May 2 • 114

Is Bigger Edit Batch Size Always Better? -- An Empirical Study on Model Editing with Llama-3

Paper • 2405.00664 • Published May 1 • 18

Clover: Regressive Lightweight Speculative Decoding with Sequential Knowledge

Paper • 2405.00263 • Published May 1 • 14

upvoted a collection 5 months ago

RAG

Collection

RAG research • 11 items • Updated Jul 22 • 2

upvoted 2 papers 5 months ago

Layer Skip: Enabling Early Exit Inference and Self-Speculative Decoding

Paper • 2404.16710 • Published Apr 25 • 57

Make Your LLM Fully Utilize the Context

Paper • 2404.16811 • Published Apr 25 • 52

upvoted an article 5 months ago

Article

The Open Medical-LLM Leaderboard: Benchmarking Large Language Models in Healthcare

Apr 19

• 99

upvoted 3 collections 5 months ago

Llama 2 Family

Collection

This collection hosts the transformers and original repos of the Llama 2 and Llama Guard releases • 13 items • Updated Aug 2 • 60

Code Llama Family

Collection

This collection hosts the transformers repos of the Code Llama release • 12 items • Updated Aug 2 • 34

Meta Llama 3

Collection

This collection hosts the transformers and original repos of the Meta Llama 3 and Llama Guard 2 releases • 5 items • Updated Aug 2 • 673

upvoted 2 papers 5 months ago

Pre-training Small Base LMs with Fewer Tokens

Paper • 2404.08634 • Published Apr 12 • 33

Rho-1: Not All Tokens Are What You Need

Paper • 2404.07965 • Published Apr 11 • 83

upvoted an article 5 months ago

Article

Binary and Scalar Embedding Quantization for Significantly Faster & Cheaper Retrieval

Mar 22

• 55

upvoted a paper 5 months ago

Stream of Search (SoS): Learning to Search in Language

Paper • 2404.03683 • Published Apr 1 • 21

upvoted 12 papers 6 months ago

The Unreasonable Ineffectiveness of the Deeper Layers

Paper • 2403.17887 • Published Mar 26 • 77

OctoPack: Instruction Tuning Code Large Language Models

Paper • 2308.07124 • Published Aug 14, 2023 • 28

Can large language models explore in-context?

Paper • 2403.15371 • Published Mar 22 • 31

CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Evaluations on HumanEval-X

Paper • 2303.17568 • Published Mar 30, 2023 • 2

Evaluating Frontier Models for Dangerous Capabilities

Paper • 2403.13793 • Published Mar 20 • 7

Recourse for reclamation: Chatting with generative language models

Paper • 2403.14467 • Published Mar 21 • 6

BurstAttention: An Efficient Distributed Attention Framework for Extremely Long Sequences

Paper • 2403.09347 • Published Mar 14 • 20

MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training

Paper • 2403.09611 • Published Mar 14 • 123

Sorted LLaMA: Unlocking the Potential of Intermediate Layers of Large Language Models for Dynamic Inference Using Sorted Fine-Tuning (SoFT)

Paper • 2309.08968 • Published Sep 16, 2023 • 22

Teaching Large Language Models to Reason with Reinforcement Learning

Paper • 2403.04642 • Published Mar 7 • 46

Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context

Paper • 2403.05530 • Published Mar 8 • 59

CogView3: Finer and Faster Text-to-Image Generation via Relay Diffusion

Paper • 2403.05121 • Published Mar 8 • 20

upvoted 7 papers 7 months ago

Design2Code: How Far Are We From Automating Front-End Engineering?

Paper • 2403.03163 • Published Mar 5 • 93

GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection

Paper • 2403.03507 • Published Mar 6 • 182

ShortGPT: Layers in Large Language Models are More Redundant Than You Expect

Paper • 2403.03853 • Published Mar 6 • 63

AtP*: An efficient and scalable method for localizing LLM behaviour to components

Paper • 2403.00745 • Published Mar 1 • 11

Learning and Leveraging World Models in Visual Representation Learning

Paper • 2403.00504 • Published Mar 1 • 29

Resonance RoPE: Improving Context Length Generalization of Large Language Models

Paper • 2403.00071 • Published Feb 29 • 22

Evaluating Very Long-Term Conversational Memory of LLM Agents

Paper • 2402.17753 • Published Feb 27 • 18

Yang Lee

AI & ML interests

Organizations

innovation64's activity

makeMoE: Implement a Sparse Mixture of Experts Language Model from Scratch

Sparse Mixture of Experts Language Model from Scratch: Extending makeMoE with Expert Capacity

Putting RL back in RLHF

Making sense of this mess

The Open Medical-LLM Leaderboard: Benchmarking Large Language Models in Healthcare

Binary and Scalar Embedding Quantization for Significantly Faster & Cheaper Retrieval