README_TEMPLATE.md

---
tags:
- llm-rs
- ggml
pipeline_tag: text-generation
license: apache-2.0
language:
- en
---

# GGML converted versions of [EleutherAI](https://huggingface.co./EleutherAI)'s Pythia models

## Description:
The *Pythia Scaling Suite* is a collection of models developed to facilitate 
interpretability research. It contains two sets of eight models of sizes 
70M, 160M, 410M, 1B, 1.4B, 2.8B, 6.9B, and 12B. For each size, there are two 
models: one trained on the Pile, and one trained on the Pile after the dataset 
has been globally deduplicated. All 8 model sizes are trained on the exact 
same data, in the exact same order. We also provide 154 intermediate 
checkpoints per model, hosted on Hugging Face as branches.

The Pythia model suite was deliberately designed to promote scientific 
research on large language models, especially interpretability research. 
Despite not centering downstream performance as a design goal, we find the 
models match or exceed the performance of 
similar and same-sized models, such as those in the OPT and GPT-Neo suites.

## Converted Models:

$MODELS$

## Usage

### Python via [llm-rs](https://github.com/LLukas22/llm-rs-python):

#### Installation
Via pip: `pip install llm-rs`

#### Run inference
```python
from llm_rs import AutoModel

#Load the model, define any model you like from the list above as the `model_file`
model = AutoModel.from_pretrained("rustformers/pythia-ggml",model_file="pythia-70m-q4_0-ggjt.bin")

#Generate
print(model.generate("The meaning of life is"))
```

### Rust via [Rustformers/llm](https://github.com/rustformers/llm): 

#### Installation
```
git clone --recurse-submodules https://github.com/rustformers/llm.git
cd llm
cargo build --release
```

#### Run inference
```
cargo run --release -- gptneox infer -m path/to/model.bin  -p "Tell me how cool the Rust programming language is:"
```