|
--- |
|
library_name: transformers |
|
license: apache-2.0 |
|
language: |
|
- en |
|
--- |
|
|
|
# test of ModernBERT2Olmo-large_1b |
|
|
|
experimental seq2seq with EncoderDecoderModel. You will need to patch `modeling_llama.py` with [this code](https://gist.github.com/pszemraj/a15219f33d94dc53a6e270c0c81360ec) for it work |
|
|
|
> [!WARNING] |
|
> WIP + output of this model is gibberish bc cross attn needs training |
|
|
|
```py |
|
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM |
|
|
|
tokenizer = AutoTokenizer.from_pretrained("pszemraj/ModernBERT2Olmo-large_1b-test") |
|
model = AutoModelForSeq2SeqLM.from_pretrained("pszemraj/ModernBERT2Olmo-large_1b-test") |
|
|
|
ARTICLE_TO_SUMMARIZE = ( |
|
"PG&E stated it scheduled the blackouts in response to forecasts for high winds " |
|
"amid dry conditions. The aim is to reduce the risk of wildfires. Nearly 800 thousand customers were " |
|
"scheduled to be affected by the shutoffs which were expected to last through at least midday tomorrow." |
|
) |
|
prompt = f"summarize dis botmon: {ARTICLE_TO_SUMMARIZE}" |
|
inputs = tokenizer(prompt, return_tensors="pt").to(model.device) |
|
|
|
# autoregressively generate summary (uses greedy decoding by default) |
|
generated_ids = model.generate( |
|
**inputs, |
|
min_new_tokens=10, |
|
max_new_tokens=100, |
|
) |
|
generated_text = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0] |
|
print(generated_text) |
|
``` |
|
|
|
|
