ClaudioItaly
commited on
Update README.md
Browse files
README.md
CHANGED
@@ -7,6 +7,8 @@ tags:
|
|
7 |
- merge
|
8 |
|
9 |
---
|
|
|
|
|
10 |
# merge
|
11 |
|
12 |
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
|
|
|
7 |
- merge
|
8 |
|
9 |
---
|
10 |
+
Changing the value of kv_count from 34 to 40 indicates an increase in the number of key-value pairs in the model. These key-value pairs are mainly used to represent attention information within neural networks, particularly in Transformer-type models such as LLaMA.
|
11 |
+
|
12 |
# merge
|
13 |
|
14 |
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
|