File size: 668 Bytes
61ec6c7 6eac43e f55d3a5 d0cc92f 61ec6c7 d0cc92f |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
---
library_name: transformers
tags: []
---
# gemma2-mitra-it-int8
This is an 8int quantized version of gemma-2-mitra-it: https://huggingface.co./buddhist-nlp/gemma-2-mitra-it
The quantization was done with llm compressor: https://github.com/vllm-project/llm-compressor
The template for prompting the model is this:
```
Please translate into <target_language>: <input_sentence> 🔽 Translation::
```
Line breaks in this model should be replaced with the '🔽' character before running the generation. '#' is used as a stop token.
## Model Details
For details on how to run this please see the gemma2-9b repository: https://huggingface.co./google/gemma-2-9b
|