icefog72 commited on
Commit
4d8d429
·
verified ·
1 Parent(s): be9c384

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +49 -48
README.md CHANGED
@@ -1,48 +1,49 @@
1
- ---
2
- base_model: []
3
- library_name: transformers
4
- tags:
5
- - mergekit
6
- - merge
7
-
8
- ---
9
- # Ice0.40-20.11-RP
10
-
11
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
12
-
13
- ## Merge Details
14
- ### Merge Method
15
-
16
- This model was merged using the SLERP merge method.
17
-
18
- ### Models Merged
19
-
20
- The following models were included in the merge:
21
- * G:\FModels\Ice0.29-06.11-RP
22
- * H:\FModels\Ice0.37-18.11-RP
23
-
24
- ### Configuration
25
-
26
- The following YAML configuration was used to produce this model:
27
-
28
- ```yaml
29
- slices:
30
- - sources:
31
- - model: H:\FModels\Ice0.37-18.11-RP
32
- layer_range: [0, 32]
33
- - model: G:\FModels\Ice0.29-06.11-RP
34
- layer_range: [0, 32]
35
-
36
- merge_method: slerp
37
- base_model: H:\FModels\Ice0.37-18.11-RP
38
- parameters:
39
- t:
40
- - filter: self_attn
41
- value: [0, 0.5, 0.3, 0.7, 1]
42
- - filter: mlp
43
- value: [1, 0.5, 0.7, 0.3, 0]
44
- - value: 0.5 # fallback for rest of tensors
45
- dtype: bfloat16
46
-
47
-
48
- ```
 
 
1
+ ---
2
+ base_model: []
3
+ license: cc-by-nc-4.0
4
+ library_name: transformers
5
+ tags:
6
+ - mergekit
7
+ - merge
8
+
9
+ ---
10
+ # Ice0.40-20.11-RP
11
+
12
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
13
+
14
+ ## Merge Details
15
+ ### Merge Method
16
+
17
+ This model was merged using the SLERP merge method.
18
+
19
+ ### Models Merged
20
+
21
+ The following models were included in the merge:
22
+ * G:\FModels\Ice0.29-06.11-RP
23
+ * H:\FModels\Ice0.37-18.11-RP
24
+
25
+ ### Configuration
26
+
27
+ The following YAML configuration was used to produce this model:
28
+
29
+ ```yaml
30
+ slices:
31
+ - sources:
32
+ - model: H:\FModels\Ice0.37-18.11-RP
33
+ layer_range: [0, 32]
34
+ - model: G:\FModels\Ice0.29-06.11-RP
35
+ layer_range: [0, 32]
36
+
37
+ merge_method: slerp
38
+ base_model: H:\FModels\Ice0.37-18.11-RP
39
+ parameters:
40
+ t:
41
+ - filter: self_attn
42
+ value: [0, 0.5, 0.3, 0.7, 1]
43
+ - filter: mlp
44
+ value: [1, 0.5, 0.7, 0.3, 0]
45
+ - value: 0.5 # fallback for rest of tensors
46
+ dtype: bfloat16
47
+
48
+
49
+ ```