LLM Pruning and Distillation in Practice: The Minitron Approach
Paper
•
2408.11796
•
Published
•
58
readings
-
-
TableBench: A Comprehensive and Complex Benchmark for Table Question Answering
Paper • 2408.09174 • Published • 52 -
To Code, or Not To Code? Exploring Impact of Code in Pre-training
Paper • 2408.10914 • Published • 42 -
Open-FinLLMs: Open Multimodal Large Language Models for Financial Applications
Paper • 2408.11878 • Published • 54 -
Law of Vision Representation in MLLMs
Paper • 2408.16357 • Published • 93 -
CURLoRA: Stable LLM Continual Fine-Tuning and Catastrophic Forgetting Mitigation
Paper • 2408.14572 • Published • 8 -
SciLitLLM: How to Adapt LLMs for Scientific Literature Understanding
Paper • 2408.15545 • Published • 35 -
LongLLaVA: Scaling Multi-modal LLMs to 1000 Images Efficiently via Hybrid Architecture
Paper • 2409.02889 • Published • 55 -
LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-context QA
Paper • 2409.02897 • Published • 45 -
Attention Heads of Large Language Models: A Survey
Paper • 2409.03752 • Published • 89 -
Guide-and-Rescale: Self-Guidance Mechanism for Effective Tuning-Free Real Image Editing
Paper • 2409.01322 • Published • 95 -
Towards a Unified View of Preference Learning for Large Language Models: A Survey
Paper • 2409.02795 • Published • 72 -
Paper Copilot: A Self-Evolving and Efficient LLM System for Personalized Academic Assistance
Paper • 2409.04593 • Published • 24 -
ProteinBench: A Holistic Evaluation of Protein Foundation Models
Paper • 2409.06744 • Published • 8 -
Qwen2.5-Coder Technical Report
Paper • 2409.12186 • Published • 140 -
Training Language Models to Self-Correct via Reinforcement Learning
Paper • 2409.12917 • Published • 136 -
HelloBench: Evaluating Long Text Generation Capabilities of Large Language Models
Paper • 2409.16191 • Published • 42 -
Making Text Embedders Few-Shot Learners
Paper • 2409.15700 • Published • 30 -
Instruction Following without Instruction Tuning
Paper • 2409.14254 • Published • 28 -
TPI-LLM: Serving 70B-scale LLMs Efficiently on Low-resource Edge Devices
Paper • 2410.00531 • Published • 30 -
From Code to Correctness: Closing the Last Mile of Code Generation with Hierarchical Debugging
Paper • 2410.01215 • Published • 30 -
Not All LLM Reasoners Are Created Equal
Paper • 2410.01748 • Published • 28 -
RATIONALYST: Pre-training Process-Supervision for Improving Reasoning
Paper • 2410.01044 • Published • 35 -
Training Language Models on Synthetic Edit Sequences Improves Code Synthesis
Paper • 2410.02749 • Published • 12 -
SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Acceleration
Paper • 2410.02367 • Published • 47 -
Addition is All You Need for Energy-efficient Language Models
Paper • 2410.00907 • Published • 145 -
Selective Attention Improves Transformer
Paper • 2410.02703 • Published • 24 -
Agent S: An Open Agentic Framework that Uses Computers Like a Human
Paper • 2410.08164 • Published • 24 -
Toward General Instruction-Following Alignment for Retrieval-Augmented Generation
Paper • 2410.09584 • Published • 47 -
A Unified View of Delta Parameter Editing in Post-Trained Large-Scale Models
Paper • 2410.13841 • Published • 16 -
HumanEval-V: Evaluating Visual Understanding and Reasoning Abilities of Large Multimodal Models Through Coding Tasks
Paper • 2410.12381 • Published • 43 -
Revealing the Barriers of Language Agents in Planning
Paper • 2410.12409 • Published • 25 -
Breaking the Memory Barrier: Near Infinite Batch Size Scaling for Contrastive Loss
Paper • 2410.17243 • Published • 89 -
Why Does the Effective Context Length of LLMs Fall Short?
Paper • 2410.18745 • Published • 17 -
Robots Pre-train Robots: Manipulation-Centric Robotic Representation from Large-Scale Robot Dataset
Paper • 2410.22325 • Published • 10 -
A Large Recurrent Action Model: xLSTM enables Fast Inference for Robotics Tasks
Paper • 2410.22391 • Published • 22 -
Both Text and Images Leaked! A Systematic Analysis of Multimodal LLM Data Contamination
Paper • 2411.03823 • Published • 44 -
Large Language Models Orchestrating Structured Reasoning Achieve Kaggle Grandmaster Level
Paper • 2411.03562 • Published • 65 -
HtmlRAG: HTML is Better Than Plain Text for Modeling Retrieved Knowledge in RAG Systems
Paper • 2411.02959 • Published • 66 -
Let the Flows Tell: Solving Graph Combinatorial Optimization Problems with GFlowNets
Paper • 2305.17010 • Published -
OpenCoder: The Open Cookbook for Top-Tier Code Large Language Models
Paper • 2411.04905 • Published • 113 -
Parameter-Efficient Fine-Tuning of Large Language Models for Unit Test Generation: An Empirical Study
Paper • 2411.02462 • Published • 9 -
Large Language Models Can Self-Improve in Long-context Reasoning
Paper • 2411.08147 • Published • 63 -
Cut Your Losses in Large-Vocabulary Language Models
Paper • 2411.09009 • Published • 44 -
ClinicalBench: Can LLMs Beat Traditional ML Models in Clinical Prediction?
Paper • 2411.06469 • Published • 17 -
SlimLM: An Efficient Small Language Model for On-Device Document Assistance
Paper • 2411.09944 • Published • 12 -
SageAttention2 Technical Report: Accurate 4 Bit Attention for Plug-and-play Inference Acceleration
Paper • 2411.10958 • Published • 52 -
Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization
Paper • 2411.10442 • Published • 72 -
Hymba: A Hybrid-head Architecture for Small Language Models
Paper • 2411.13676 • Published • 41 -
Natural Language Reinforcement Learning
Paper • 2411.14251 • Published • 28 -
Cautious Optimizers: Improving Training with One Line of Code
Paper • 2411.16085 • Published • 15 -
Predicting Emergent Capabilities by Finetuning
Paper • 2411.16035 • Published • 6 -
Star Attention: Efficient LLM Inference over Long Sequences
Paper • 2411.17116 • Published • 49 -
o1-Coder: an o1 Replication for Coding
Paper • 2412.00154 • Published • 43 -
Critical Tokens Matter: Token-Level Contrastive Estimation Enhence LLM's Reasoning Capability
Paper • 2411.19943 • Published • 57 -
VisionZip: Longer is Better but Not Necessary in Vision Language Models
Paper • 2412.04467 • Published • 105 -
Code-as-Monitor: Constraint-aware Visual Programming for Reactive and Proactive Robotic Failure Detection
Paper • 2412.04455 • Published • 37 -
Personalized Multimodal Large Language Models: A Survey
Paper • 2412.02142 • Published • 13 -
Evaluating Language Models as Synthetic Data Generators
Paper • 2412.03679 • Published • 46 -
Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling
Paper • 2412.05271 • Published • 129 -
MAmmoTH-VL: Eliciting Multimodal Reasoning with Instruction Tuning at Scale
Paper • 2412.05237 • Published • 47 -
EXAONE 3.5: Series of Large Language Models for Real-world Use Cases
Paper • 2412.04862 • Published • 50 -
Moto: Latent Motion Token as the Bridging Language for Robot Manipulation
Paper • 2412.04445 • Published • 21 -
Evaluating and Aligning CodeLLMs on Human Preference
Paper • 2412.05210 • Published • 47 -
POINTS1.5: Building a Vision-Language Model towards Real World Applications
Paper • 2412.08443 • Published • 38 -
Phi-4 Technical Report
Paper • 2412.08905 • Published • 105 -
InternLM-XComposer2.5-OmniLive: A Comprehensive Multimodal System for Long-term Streaming Video and Audio Interactions
Paper • 2412.09596 • Published • 93 -
GenEx: Generating an Explorable World
Paper • 2412.09624 • Published • 89 -
TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks
Paper • 2412.14161 • Published • 50 -
Qwen2.5 Technical Report
Paper • 2412.15115 • Published • 340 -
LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks
Paper • 2412.15204 • Published • 33 -
How to Synthesize Text Data without Model Collapse?
Paper • 2412.14689 • Published • 48 -
Offline Reinforcement Learning for LLM Multi-Step Reasoning
Paper • 2412.16145 • Published • 38 -
SCOPE: Optimizing Key-Value Cache Compression in Long-context Generation
Paper • 2412.13649 • Published • 20 -
B-STaR: Monitoring and Balancing Exploration and Exploitation in Self-Taught Reasoners
Paper • 2412.17256 • Published • 45 -
RobustFT: Robust Supervised Fine-tuning for Large Language Models under Noisy Response
Paper • 2412.14922 • Published • 85 -
Diving into Self-Evolving Training for Multimodal Reasoning
Paper • 2412.17451 • Published • 42 -
Revisiting In-Context Learning with Long Context Language Models
Paper • 2412.16926 • Published • 29 -
Outcome-Refining Process Supervision for Code Generation
Paper • 2412.15118 • Published • 19 -
DRT-o1: Optimized Deep Reasoning Translation via Long Chain-of-Thought
Paper • 2412.17498 • Published • 21 -
NILE: Internal Consistency Alignment in Large Language Models
Paper • 2412.16686 • Published • 8 -
LearnLM: Improving Gemini for Learning
Paper • 2412.16429 • Published • 22 -
PC Agent: While You Sleep, AI Works -- A Cognitive Journey into Digital World
Paper • 2412.17589 • Published • 12 -
3DGraphLLM: Combining Semantic Graphs and Large Language Models for 3D Scene Understanding
Paper • 2412.18450 • Published • 32 -
Fourier Position Embedding: Enhancing Attention's Periodic Extension for Length Generalization
Paper • 2412.17739 • Published • 40 -
ReMoE: Fully Differentiable Mixture-of-Experts with ReLU Routing
Paper • 2412.14711 • Published • 16 -
Ensembling Large Language Models with Process Reward-Guided Tree Search for Better Complex Reasoning
Paper • 2412.15797 • Published • 17 -
YuLan-Mini: An Open Data-efficient Language Model
Paper • 2412.17743 • Published • 64 -
Molar: Multimodal LLMs with Collaborative Filtering Alignment for Enhanced Sequential Recommendation
Paper • 2412.18176 • Published • 15 -
MMFactory: A Universal Solution Search Engine for Vision-Language Tasks
Paper • 2412.18072 • Published • 17 -
Explanatory Instructions: Towards Unified Vision Tasks Understanding and Zero-shot Generalization
Paper • 2412.18525 • Published • 70 -
Efficiently Serving LLM Reasoning Programs with Certaindex
Paper • 2412.20993 • Published • 35 -
HumanEval Pro and MBPP Pro: Evaluating Large Language Models on Self-invoking Code Generation
Paper • 2412.21199 • Published • 13 -
OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesis
Paper • 2412.19723 • Published • 81 -
2.5 Years in Class: A Multimodal Textbook for Vision-Language Pretraining
Paper • 2501.00958 • Published • 97 -
CodeElo: Benchmarking Competition-level Code Generation of LLMs with Human-comparable Elo Ratings
Paper • 2501.01257 • Published • 48 -
Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Models
Paper • 2501.01423 • Published • 36 -
ProgCo: Program Helps Self-Correction of Large Language Models
Paper • 2501.01264 • Published • 25 -
STAR: Spatial-Temporal Augmentation with Text-to-Video Models for Real-World Video Super-Resolution
Paper • 2501.02976 • Published • 52 -
BoostStep: Boosting mathematical capability of Large Language Models via improved single-step reasoning
Paper • 2501.03226 • Published • 37 -
Test-time Computing: from System-1 Thinking to System-2 Thinking
Paper • 2501.02497 • Published • 41 -
REINFORCE++: A Simple and Efficient Approach for Aligning Large Language Models
Paper • 2501.03262 • Published • 86 -
MotionBench: Benchmarking and Improving Fine-grained Video Motion Understanding for Vision Language Models
Paper • 2501.02955 • Published • 40 -
LLaVA-Mini: Efficient Image and Video Large Multimodal Models with One Vision Token
Paper • 2501.03895 • Published • 48 -
Cosmos World Foundation Model Platform for Physical AI
Paper • 2501.03575 • Published • 66 -
PPTAgent: Generating and Evaluating Presentations Beyond Text-to-Slides
Paper • 2501.03936 • Published • 19 -
An Empirical Study of Autoregressive Pre-training from Videos
Paper • 2501.05453 • Published • 36 -
Enhancing Human-Like Responses in Large Language Models
Paper • 2501.05032 • Published • 49 -
SWE-Fixer: Training Open-Source LLMs for Effective and Efficient GitHub Issue Resolution
Paper • 2501.05040 • Published • 14 -
LlamaV-o1: Rethinking Step-by-step Visual Reasoning in LLMs
Paper • 2501.06186 • Published • 59 -
VideoRAG: Retrieval-Augmented Generation over Video Corpus
Paper • 2501.05874 • Published • 66 -
OVO-Bench: How Far is Your Video-LLMs from Real-World Online Video Understanding?
Paper • 2501.05510 • Published • 37 -
The Lessons of Developing Process Reward Models in Mathematical Reasoning
Paper • 2501.07301 • Published • 85 -
Tensor Product Attention Is All You Need
Paper • 2501.06425 • Published • 75 -
Transformer^2: Self-adaptive LLMs
Paper • 2501.06252 • Published • 50 -
WebWalker: Benchmarking LLMs in Web Traversal
Paper • 2501.07572 • Published • 19 -
O1 Replication Journey -- Part 3: Inference-time Scaling for Medical Reasoning
Paper • 2501.06458 • Published • 29 -
Towards Best Practices for Open Datasets for LLM Training
Paper • 2501.08365 • Published • 47 -
MMDocIR: Benchmarking Multi-Modal Retrieval for Long Documents
Paper • 2501.08828 • Published • 28 -
Inference-Time Scaling for Diffusion Models beyond Scaling Denoising Steps
Paper • 2501.09732 • Published • 64 -
Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Models
Paper • 2501.09686 • Published • 35 -
FAST: Efficient Action Tokenization for Vision-Language-Action Models
Paper • 2501.09747 • Published • 22 -
Evolving Deeper LLM Thinking
Paper • 2501.09891 • Published • 93 -
PaSa: An LLM Agent for Comprehensive Academic Paper Search
Paper • 2501.10120 • Published • 36 -
Agent-R: Training Language Model Agents to Reflect via Iterative Self-Training
Paper • 2501.11425 • Published • 68 -
Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of-Expert Models
Paper • 2501.11873 • Published • 57 -
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
Paper • 2501.12948 • Published • 85 -
Test-Time Preference Optimization: On-the-Fly Alignment via Iterative Textual Feedback
Paper • 2501.12895 • Published • 39 -
VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understanding
Paper • 2501.13106 • Published • 35 -
Kimi k1.5: Scaling Reinforcement Learning with LLMs
Paper • 2501.12599 • Published • 29 -
Autonomy-of-Experts Models
Paper • 2501.13074 • Published • 26