File size: 1,401 Bytes
eda73a2 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 |
## Approach
This model of [Mamba architecture](https://arxiv.org/abs/2312.00752) has been pre-trained on approximately 400B tokens of Chinese and English corpora.
## Usage
```python
import torch
from mamba_ssm.models.mixer_seq_simple import MambaLMHeadModel
from transformers import AutoTokenizer
repo_id = 'mamba-1.4b-aquila-400b'
device = f"cuda:0"
model = MambaLMHeadModel.from_pretrained(repo_id, dtype=torch.bfloat16, device=device)
model.eval()
tokenizer = AutoTokenizer.from_pretrained(repo_id)
prompt = "The Spring Festival is"
tokens = tokenizer.encode_plus(prompt, truncation=False)["input_ids"]
tokens = torch.tensor(tokens)[None,].to(device)
with torch.no_grad():
input_length = len(tokens[0])
out_ids = model.generate(input_ids=tokens, max_length=input_length+200, temperature=1.0, top_p=0.95, eos_token_id=tokenizer.eos_token_id, cg=True, top_k=15)
out_ids = out_ids[0][input_length:].cpu().numpy()
out_text = tokenizer.decode(out_ids.tolist())
print(out_text)
```
> the most important festival of the year for the Chinese people. It usually comes in January or February and it takes about 15 days to prepare for it.
## References
The Mamba architecture was introduced in [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://arxiv.org/abs/2312.00752).
The official implementation is here: https://github.com/state-spaces/mamba/tree/main
|