|
--- |
|
library_name: transformers |
|
license: apache-2.0 |
|
language: |
|
- ko |
|
--- |
|
|
|
# Model Card for Model ID |
|
|
|
|
|
## Model Details |
|
|
|
### Model Description |
|
|
|
The Gemma Self-Attention Merged model is a large language model created by merging the self-attention layers of an [English-based Gemma 7B model](https://huggingface.co./google/gemma-1.1-7b-it) and a [Korean-based Gemma 7B model](beomi/gemma-ko-7b). This merger allows the model to leverage the capabilities of both the English and Korean models, resulting in a more versatile and capable language model that can perform well on tasks involving both English and Korean text. |
|
|
|
The key features of this merged model include: |
|
|
|
- Increased self-attention capacity with doubled number of attention heads |
|
- Ability to handle both English and Korean language input |
|
- Potential for improved performance on a wide range of natural language processing tasks |
|
|
|
#### Chat template |
|
|
|
**system:** system message... |
|
**B:** user message... |
|
**A:** assistant message... |
|
|
|
### Model Sources |
|
|
|
- **Repository:** https://github.com/lcw99/merge-gemma-attn.git |
