ClaudioItaly commited on
Commit
2653389
·
verified ·
1 Parent(s): e8e8946

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -0
README.md CHANGED
@@ -7,6 +7,8 @@ tags:
7
  - merge
8
 
9
  ---
 
 
10
  # merge
11
 
12
  This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
 
7
  - merge
8
 
9
  ---
10
+ Changing the value of kv_count from 34 to 40 indicates an increase in the number of key-value pairs in the model. These key-value pairs are mainly used to represent attention information within neural networks, particularly in Transformer-type models such as LLaMA.
11
+
12
  # merge
13
 
14
  This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).