Models
Datasets
Spaces
Posts
Docs
Pricing
Log In
Sign Up

Collections

Discover the best community collections!

Collections including paper arxiv:2404.07129

Interpretability

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10 • 3

INSTRUCTEVAL: Towards Holistic Evaluation of Instruction-Tuned Large Language Models

Paper • 2306.04757 • Published Jun 7, 2023 • 6
Evaluating Instruction-Tuned Large Language Models on Code Comprehension and Generation

Paper • 2308.01240 • Published Aug 2, 2023 • 2
Can Large Language Models Understand Real-World Complex Instructions?

Paper • 2309.09150 • Published Sep 17, 2023 • 2
Evaluating the Instruction-Following Robustness of Large Language Models to Prompt Injection

Paper • 2308.10819 • Published Aug 17, 2023

Papers - ICL - Residual Head Hypothesis

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10 • 3

Papers - XAI - Attention - LayerNorm

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10 • 3

Papers - ICL - Phase Change Delay - Large Vocabulary Size

Larger vocab is better compression, but may result in longer training ICL phase change delays due to the slower Induction Head Copy Subcircuit (C)

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10 • 3

Papers - ICL - Phase Change - Delay - Classes and Labels

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10 • 3

Papers - ICL - Induction Head - Copy vs QK Match

See figure 6: Classes vs labels in columns B and C. Subcircuit B delays phase change on number classes vs C delays on number of labels (dramatically)

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10 • 3

Papers - Training - ICL - Induction Circuit Evolution

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10 • 3

Papers - ICL - Induction Circuit - Data Dependent Learning

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10 • 3

Papers - ICL - Induction Head - Num Labels vs Classes - Loss

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10 • 3

Previous
1
2
3
4
Next

Company

© Hugging Face

TOS Privacy About Jobs

Website

Models Datasets Spaces Pricing Docs