---
language: en
library_name: mlsae
license: mit
tags:
  - model_hub_mixin
  - pytorch_model_hub_mixin
datasets:
  - monology/pile-uncopyrighted
---

# mlsae-pythia-70m-deduped-x4-k32

A Multi-Layer Sparse Autoencoder (MLSAE) trained on the residual stream
activation vectors from every layer of
[EleutherAI/pythia-70m-deduped](https://huggingface.co./EleutherAI/pythia-70m-deduped)
with an expansion factor of 4 and k = 32, over 1 billion tokens from
[monology/pile-uncopyrighted](https://huggingface.co./datasets/monology/pile-uncopyrighted).

For more details, see:

- Paper: <https://arxiv.org/abs/2409.04185>
- GitHub repository: <https://github.com/tim-lawson/mlsae>
- Weights & Biases project: <https://wandb.ai/timlawson-/mlsae>