qinluo (qinluo)

upvoted a paper 2 days ago

V-STaR: Training Verifiers for Self-Taught Reasoners

Paper • 2402.06457 • Published Feb 9 • 8

upvoted a paper 8 days ago

LongRecipe: Recipe for Efficient Long Context Generalization in Large Languge Models

Paper • 2409.00509 • Published 20 days ago • 38

upvoted a paper 17 days ago

RegMix: Data Mixture as Regression for Language Model Pre-training

Paper • 2407.01492 • Published Jul 1 • 33

upvoted an article about 2 months ago

Article

SmolLM - blazingly fast and remarkably powerful

Jul 16

• 242

upvoted a paper 3 months ago

The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale

Paper • 2406.17557 • Published Jun 25 • 84

upvoted a collection 3 months ago

AnyTaskTune-Psychology

Collection

Elevating Domain-Specific Model Performance with Precision Task-Specific Fine-Tuning • 3 items • Updated Jun 7 • 2

upvoted 2 papers 4 months ago

Xmodel-LM Technical Report

Paper • 2406.02856 • Published Jun 5 • 7

FP6-LLM: Efficiently Serving Large Language Models Through FP6-Centric Algorithm-System Co-Design

Paper • 2401.14112 • Published Jan 25 • 17

upvoted 3 papers 5 months ago

Decoding Compressed Trust: Scrutinizing the Trustworthiness of Efficient LLMs Under Compression

Paper • 2403.15447 • Published Mar 18 • 15

WildChat: 1M ChatGPT Interaction Logs in the Wild

Paper • 2405.01470 • Published May 2 • 59

KAN: Kolmogorov-Arnold Networks

Paper • 2404.19756 • Published Apr 30 • 108

upvoted 3 articles 5 months ago

Article

⚗️ 🧑🏼‍🌾 Let's grow some Domain Specific Datasets together

By

•

Apr 29

• 28

Article

🦙⚗️ Using Llama3 and distilabel to build fine-tuning datasets

By

•

Jun 4

• 67

Article

Fine-tune Llama 3 with ORPO

By

•

Apr 22

• 221

upvoted 2 papers 5 months ago

Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Laws

Paper • 2404.05405 • Published Apr 8 • 7

Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction

Paper • 2404.02905 • Published Apr 3 • 63

upvoted 7 papers 6 months ago

Mixture-of-Depths: Dynamically allocating compute in transformer-based language models

Paper • 2404.02258 • Published Apr 2 • 103

Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning

Paper • 2402.06619 • Published Feb 9 • 52

Synthetic Data (Almost) from Scratch: Generalized Instruction Tuning for Language Models

Paper • 2402.13064 • Published Feb 20 • 46

What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning

Paper • 2312.15685 • Published Dec 25, 2023 • 17

Instruction-tuned Language Models are Better Knowledge Learners

Paper • 2402.12847 • Published Feb 20 • 24

Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models

Paper • 2401.04658 • Published Jan 9 • 24

ORPO: Monolithic Preference Optimization without Reference Model

Paper • 2403.07691 • Published Mar 12 • 59

upvoted a collection 6 months ago

Common Corpus

Collection

The largest public domain dataset for training LLMs. • 27 items • Updated Jul 17 • 111

upvoted a paper 6 months ago

GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection

Paper • 2403.03507 • Published Mar 6 • 182

upvoted a paper 9 months ago

LLM360: Towards Fully Transparent Open-Source LLMs

Paper • 2312.06550 • Published Dec 11, 2023 • 56

upvoted a paper 10 months ago

GPT-4V in Wonderland: Large Multimodal Models for Zero-Shot Smartphone GUI Navigation

Paper • 2311.07562 • Published Nov 13, 2023 • 12

upvoted a paper 11 months ago

Zephyr: Direct Distillation of LM Alignment

Paper • 2310.16944 • Published Oct 25, 2023 • 120

upvoted 2 papers 12 months ago

Language Modeling Is Compression

Paper • 2309.10668 • Published Sep 19, 2023 • 82

CulturaX: A Cleaned, Enormous, and Multilingual Dataset for Large Language Models in 167 Languages

Paper • 2309.09400 • Published Sep 17, 2023 • 82

upvoted 8 papers about 1 year ago

FLM-101B: An Open LLM and How to Train It with $100K Budget

Paper • 2309.03852 • Published Sep 7, 2023 • 43

Textbooks Are All You Need II: phi-1.5 technical report

Paper • 2309.05463 • Published Sep 11, 2023 • 86

Skills-in-Context Prompting: Unlocking Compositionality in Large Language Models

Paper • 2308.00304 • Published Aug 1, 2023 • 23

qinluo

AI & ML interests

Organizations

qinluo's activity

SmolLM - blazingly fast and remarkably powerful

⚗️ 🧑🏼‍🌾 Let's grow some Domain Specific Datasets together

🦙⚗️ Using Llama3 and distilabel to build fine-tuning datasets

Fine-tune Llama 3 with ORPO