DavidAU commited on
Commit
ef79ec2
1 Parent(s): 1425f36

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +79 -75
README.md CHANGED
@@ -1,75 +1,79 @@
1
- ---
2
- base_model: []
3
- library_name: transformers
4
- tags:
5
- - mergekit
6
- - merge
7
-
8
- ---
9
- # L3-Lumimaid-12.2B-v0.1-OAS-Instruct - Float32
10
-
11
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
12
-
13
- ## Merge Details
14
- ### Merge Method
15
-
16
- This model was merged using the passthrough merge method.
17
-
18
- ### Models Merged
19
-
20
- The following models were included in the merge:
21
- * G:/7B/Meta-Llama-3-8B-Instruct
22
- * G:/7B/Llama-3-Lumimaid-8B-v0.1-OAS
23
-
24
- ### Configuration
25
-
26
- The following YAML configuration was used to produce this model:
27
-
28
- ```yaml
29
- slices:
30
- - sources:
31
- - model: G:/7B/Meta-Llama-3-8B-Instruct
32
- layer_range: [0, 12]
33
- - sources:
34
- - model: G:/7B/Llama-3-Lumimaid-8B-v0.1-OAS
35
- layer_range: [6, 19]
36
- parameters:
37
- scale:
38
- - filter: o_proj
39
- value: 1
40
- - filter: down_proj
41
- value: 1
42
- - value: 1
43
- - sources:
44
- - model: G:/7B/Meta-Llama-3-8B-Instruct
45
- layer_range: [12, 18]
46
- parameters:
47
- scale:
48
- - filter: o_proj
49
- value: .5
50
- - filter: down_proj
51
- value: .5
52
- - value: 1
53
- - sources:
54
- - model: G:/7B/Meta-Llama-3-8B-Instruct
55
- layer_range: [18, 25]
56
- parameters:
57
- scale:
58
- - filter: o_proj
59
- value: .75
60
- - filter: down_proj
61
- value: .75
62
- - value: 1
63
- - sources:
64
- - model: G:/7B/Llama-3-Lumimaid-8B-v0.1-OAS
65
- layer_range: [19, 32]
66
- parameters:
67
- scale:
68
- - filter: o_proj
69
- value: 1
70
- - filter: down_proj
71
- value: 1
72
- - value: 1
73
- merge_method: passthrough
74
- dtype: float32
75
- ```
 
 
 
 
 
1
+ ---
2
+ base_model: []
3
+ library_name: transformers
4
+ tags:
5
+ - mergekit
6
+ - merge
7
+
8
+ ---
9
+ # L3-Lumimaid-12.2B-v0.1-OAS-Instruct - Float32
10
+
11
+ For GGufs and full model card please go to:
12
+
13
+ [ https://huggingface.co/DavidAU/L3-Lumimaid-v0.1-OAS-12.2B-INSTRUCT-ULTRA-F32-GGUF ]
14
+
15
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
16
+
17
+ ## Merge Details
18
+ ### Merge Method
19
+
20
+ This model was merged using the passthrough merge method.
21
+
22
+ ### Models Merged
23
+
24
+ The following models were included in the merge:
25
+ * G:/7B/Meta-Llama-3-8B-Instruct
26
+ * G:/7B/Llama-3-Lumimaid-8B-v0.1-OAS
27
+
28
+ ### Configuration
29
+
30
+ The following YAML configuration was used to produce this model:
31
+
32
+ ```yaml
33
+ slices:
34
+ - sources:
35
+ - model: G:/7B/Meta-Llama-3-8B-Instruct
36
+ layer_range: [0, 12]
37
+ - sources:
38
+ - model: G:/7B/Llama-3-Lumimaid-8B-v0.1-OAS
39
+ layer_range: [6, 19]
40
+ parameters:
41
+ scale:
42
+ - filter: o_proj
43
+ value: 1
44
+ - filter: down_proj
45
+ value: 1
46
+ - value: 1
47
+ - sources:
48
+ - model: G:/7B/Meta-Llama-3-8B-Instruct
49
+ layer_range: [12, 18]
50
+ parameters:
51
+ scale:
52
+ - filter: o_proj
53
+ value: .5
54
+ - filter: down_proj
55
+ value: .5
56
+ - value: 1
57
+ - sources:
58
+ - model: G:/7B/Meta-Llama-3-8B-Instruct
59
+ layer_range: [18, 25]
60
+ parameters:
61
+ scale:
62
+ - filter: o_proj
63
+ value: .75
64
+ - filter: down_proj
65
+ value: .75
66
+ - value: 1
67
+ - sources:
68
+ - model: G:/7B/Llama-3-Lumimaid-8B-v0.1-OAS
69
+ layer_range: [19, 32]
70
+ parameters:
71
+ scale:
72
+ - filter: o_proj
73
+ value: 1
74
+ - filter: down_proj
75
+ value: 1
76
+ - value: 1
77
+ merge_method: passthrough
78
+ dtype: float32
79
+ ```