---
library_name: transformers
license: apache-2.0
language:
- ko
---

# Model Card for Model ID


## Model Details

### Model Description

The Gemma Self-Attention Merged model is a large language model created by merging the self-attention layers of an [English-based Gemma 7B model](https://huggingface.co./google/gemma-1.1-7b-it) and a [Korean-based Gemma 7B model](beomi/gemma-ko-7b). This merger allows the model to leverage the capabilities of both the English and Korean models, resulting in a more versatile and capable language model that can perform well on tasks involving both English and Korean text.

The key features of this merged model include:

- Increased self-attention capacity with doubled number of attention heads
- Ability to handle both English and Korean language input
- Potential for improved performance on a wide range of natural language processing tasks

#### Chat template

**system:** system message...  
**B:** user message...  
**A:** assistant message...  

### Model Sources

- **Repository:** https://github.com/lcw99/merge-gemma-attn.git