Hugging Face's logo

FluffyKaeloky
/
Luminum-v0.1-123B

Text Generation
Transformers
Safetensors
mistral
mergekit
Merge
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
5
FluffyKaeloky commited on
Commit
4d02594
•
1 Parent(s): c9b6c48

Upload folder using huggingface_hub

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. LuminumMistral-123B_measurements.json +0 -0
  2. README.md +45 -0
  3. config.json +26 -0
  4. mergekit_config.yml +16 -0
  5. model-00001-of-00051.safetensors +3 -0
  6. model-00002-of-00051.safetensors +3 -0
  7. model-00003-of-00051.safetensors +3 -0
  8. model-00004-of-00051.safetensors +3 -0
  9. model-00005-of-00051.safetensors +3 -0
  10. model-00006-of-00051.safetensors +3 -0
  11. model-00007-of-00051.safetensors +3 -0
  12. model-00008-of-00051.safetensors +3 -0
  13. model-00009-of-00051.safetensors +3 -0
  14. model-00010-of-00051.safetensors +3 -0
  15. model-00011-of-00051.safetensors +3 -0
  16. model-00012-of-00051.safetensors +3 -0
  17. model-00013-of-00051.safetensors +3 -0
  18. model-00014-of-00051.safetensors +3 -0
  19. model-00015-of-00051.safetensors +3 -0
  20. model-00016-of-00051.safetensors +3 -0
  21. model-00017-of-00051.safetensors +3 -0
  22. model-00018-of-00051.safetensors +3 -0
  23. model-00019-of-00051.safetensors +3 -0
  24. model-00020-of-00051.safetensors +3 -0
  25. model-00021-of-00051.safetensors +3 -0
  26. model-00022-of-00051.safetensors +3 -0
  27. model-00023-of-00051.safetensors +3 -0
  28. model-00024-of-00051.safetensors +3 -0
  29. model-00025-of-00051.safetensors +3 -0
  30. model-00026-of-00051.safetensors +3 -0
  31. model-00027-of-00051.safetensors +3 -0
  32. model-00028-of-00051.safetensors +3 -0
  33. model-00029-of-00051.safetensors +3 -0
  34. model-00030-of-00051.safetensors +3 -0
  35. model-00031-of-00051.safetensors +3 -0
  36. model-00032-of-00051.safetensors +3 -0
  37. model-00033-of-00051.safetensors +3 -0
  38. model-00034-of-00051.safetensors +3 -0
  39. model-00035-of-00051.safetensors +3 -0
  40. model-00036-of-00051.safetensors +3 -0
  41. model-00037-of-00051.safetensors +3 -0
  42. model-00038-of-00051.safetensors +3 -0
  43. model-00039-of-00051.safetensors +3 -0
  44. model-00040-of-00051.safetensors +3 -0
  45. model-00041-of-00051.safetensors +3 -0
  46. model-00042-of-00051.safetensors +3 -0
  47. model-00043-of-00051.safetensors +3 -0
  48. model-00044-of-00051.safetensors +3 -0
  49. model-00045-of-00051.safetensors +3 -0
  50. model-00046-of-00051.safetensors +3 -0
LuminumMistral-123B_measurements.json ADDED
The diff for this file is too large to render. See raw diff
 
README.md ADDED
@@ -0,0 +1,45 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: []
3
+ library_name: transformers
4
+ tags:
5
+ - mergekit
6
+ - merge
7
+
8
+ ---
9
+ # LuminumMistral-123B
10
+
11
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
12
+
13
+ ## Merge Details
14
+ ### Merge Method
15
+
16
+ This model was merged using the della_linear merge method using mistralaiMistral-Large-Instruct-2407 as a base.
17
+
18
+ ### Models Merged
19
+
20
+ The following models were included in the merge:
21
+ * NeverSleepLumimaid-v0.2-123B
22
+ * anthracite-orgmagnum-v2-123b
23
+
24
+ ### Configuration
25
+
26
+ The following YAML configuration was used to produce this model:
27
+
28
+ ```yaml
29
+ models:
30
+ - model: anthracite-orgmagnum-v2-123b
31
+ parameters:
32
+ weight: 0.19
33
+ density: 0.5
34
+ - model: NeverSleepLumimaid-v0.2-123B
35
+ parameters:
36
+ weight: 0.34
37
+ density: 0.8
38
+ merge_method: della_linear
39
+ base_model: mistralaiMistral-Large-Instruct-2407
40
+ parameters:
41
+ epsilon: 0.05
42
+ lambda: 1
43
+ int8_mask: true
44
+ dtype: bfloat16
45
+ ```
config.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "MistralForCausalLM"
4
+ ],
5
+ "attention_dropout": 0.0,
6
+ "bos_token_id": 1,
7
+ "eos_token_id": 2,
8
+ "head_dim": 128,
9
+ "hidden_act": "silu",
10
+ "hidden_size": 12288,
11
+ "initializer_range": 0.02,
12
+ "intermediate_size": 28672,
13
+ "max_position_embeddings": 131072,
14
+ "model_type": "mistral",
15
+ "num_attention_heads": 96,
16
+ "num_hidden_layers": 88,
17
+ "num_key_value_heads": 8,
18
+ "rms_norm_eps": 1e-05,
19
+ "rope_theta": 1000000.0,
20
+ "sliding_window": null,
21
+ "tie_word_embeddings": false,
22
+ "torch_dtype": "bfloat16",
23
+ "transformers_version": "4.44.2",
24
+ "use_cache": true,
25
+ "vocab_size": 32768
26
+ }
mergekit_config.yml ADDED
@@ -0,0 +1,16 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ models:
2
+ - model: S:\Models\anthracite-orgmagnum-v2-123b
3
+ parameters:
4
+ weight: 0.19
5
+ density: 0.5
6
+ - model: S:\Models\NeverSleepLumimaid-v0.2-123B
7
+ parameters:
8
+ weight: 0.34
9
+ density: 0.8
10
+ merge_method: della_linear
11
+ base_model: S:\Models\mistralaiMistral-Large-Instruct-2407
12
+ parameters:
13
+ epsilon: 0.05
14
+ lambda: 1
15
+ int8_mask: true
16
+ dtype: bfloat16
model-00001-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4f975d5366a3cd150555fe4e5262a8ada7cc625dd9e3de46b37abf61821a8bd8
3
+ size 4378928504
model-00002-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fa5a07164d554688cc1d924729b9b80e197e3465553854211fe04ae35d135cae
3
+ size 4907411088
model-00003-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:75f188b46edbfb511aa2d88d65ce8edb45165682b92941318d96b00e35e3ceff
3
+ size 4806747904
model-00004-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:db6da55820c146649a3a84c227bf43d545d503d3b0a03c566c2d3eb31c77baea
3
+ size 4831938544
model-00005-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c3d05b4276d764f77da14197a546388450ff3e0e8bdec580bb79099a51c53802
3
+ size 4831938552
model-00006-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:52c30cbb3722a4fc22f8cc2b2cb2aef1837a5383c4d85a6f798d212acc4ca4dd
3
+ size 4907411096
model-00007-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:10fc6dd3c1408001eeba43460d2314a1b3e64c0475cfa716bac9b0f70295ebc3
3
+ size 4806747904
model-00008-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:183e4b5bbc3e22f84e15b76264f165a7b5cf0d141b0ae47ff463f7671197f224
3
+ size 4831938536
model-00009-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:417a8c788394527cf7d75f9da5991e81c2b95649cd46b6c4391f7efe771eae06
3
+ size 4831938552
model-00010-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:06b1ce4107ef422b617db252a9665bb84fd7d3420bee97a75ec1d4f58fade1ff
3
+ size 4907411096
model-00011-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4c87aa6e734b4251600d90d04c6d79fd7083a0e39f2c45c43e7cfca3b5a0192c
3
+ size 4806747904
model-00012-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b37df1434d613aef55a05dce24e0dfd5020d7b471b0342a95977625efd4f18a5
3
+ size 4831938544
model-00013-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8cd9280b533111ca73e0dde991d7938ec18a94ba4275969e38e2a3a5cc86265c
3
+ size 4831938552
model-00014-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e4f57a39fbb4457f056793235c093359b152869a16a21ebc0d138dc7247fb07c
3
+ size 4907411088
model-00015-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:93265239062df03fe77ba8297819bb4f8fd5e19546a4c569008787ec6f398b6c
3
+ size 4806747904
model-00016-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0eb7579f654223f2c76f8bca39fe8ff76fb68ffc2ed731a5ea1f5a5366075ab9
3
+ size 4831938544
model-00017-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1d1d6a83c5a9a9f94c819b7a42b538e0f92eaa8bb1780cef9ba2c6147c90c4ca
3
+ size 4831938552
model-00018-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:63e993e1e7d26ea347cdf77c9a15276e5ff823928f02be41c3aee558f7c318b6
3
+ size 4907411096
model-00019-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:370163ba0dec797814211e51f82f9f0934c3506afda7cd4c17f11f2db9f880be
3
+ size 4806747904
model-00020-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:eaff1ae32ddf1bd4e6dbc49691fb5f97df2f5624d6f6e350663ba70cb99a4402
3
+ size 4831938544
model-00021-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2896d4d94eb18ed918959dc8a721c4cef26e046e0ca96cccdec64e7eb9e0649e
3
+ size 4831938544
model-00022-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5544deb7905feae1e6c5ccd47bbc622a91cbd8858e44259585d59bfa382f7b4c
3
+ size 4907411096
model-00023-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8a8b1eabf79731d8cff397e359012cd9e50636e49b95cdd0a25f3a3bb0c98914
3
+ size 4806747904
model-00024-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ef78537f1bfacaced8d1b6acca01c27da4fdd2ad6d0a7e10a8924873ac76179c
3
+ size 4831938544
model-00025-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3fd694cd862e026253320e91cfd25035d8baeba81a89570cbd696014eec4aa9f
3
+ size 4831938552
model-00026-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5866c9cf376dca261e4634e52653deee1140a950f198c13868d45e9e3d879aa0
3
+ size 4907411096
model-00027-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:49a952aae5aa076029255f9cbdc3e5e2fea87f5da9c3cdf5bbba0b7e607ef8a0
3
+ size 4806747896
model-00028-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:708dc1aa0ec262f1bfb3f945fe620289fa65911bf9ef12073a14790482193ab9
3
+ size 4831938544
model-00029-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3db28c97d6143453fcd456e3f70fa754e98cb2f6774c5db2c78712735cbddd6d
3
+ size 4831938552
model-00030-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5e5138383969489752d6c70f2be00c11eaf0036c269fd5d1108108186eebeb63
3
+ size 4907411096
model-00031-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5d45c923714c965d0c77289a2524b42cbee163001338ed4c46d5f68c794a66a9
3
+ size 4806747904
model-00032-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e13573093d864335b4d3e0494399cab9ce9458fc337c0b2b74f2d8a1857382e6
3
+ size 4831938544
model-00033-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d6bbb3c1926c3343050706a7048b0651099252bd09b683b28164be0b529a8bf4
3
+ size 4831938544
model-00034-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9ad95b4767f68bd6c2c5cbcad72f2bd460ae51e247824a10f40ee1a91a1783c4
3
+ size 4907411096
model-00035-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a5fda23f1592e95848b50401bc960d949cf52a8482355e6dbce062a7074b8926
3
+ size 4806747904
model-00036-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:38016b3eb34941c242541e6ecd1e950ec9614a74ac38d032c4b21c877c6dd082
3
+ size 4831938544
model-00037-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0dd159268e59f9d357ebe89855dcd9f02b40e84a058122a99e05994ef1bcc6b6
3
+ size 4831938552
model-00038-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2f8fb74a82dd7028149565249dcfc0dee6c1fc0f2e33f99a6f518829d824a564
3
+ size 4907411096
model-00039-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:452dd2d7a67ccb2a37cabecc78df5e9c680badf528ffd3a422b126bfe7672ae0
3
+ size 4806747904
model-00040-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8418d01ed7fc60ad4a46b1c75de5a1287510583f3c61f849c54b4e8c0cca30e1
3
+ size 4831938544
model-00041-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:92fb34e9b9b951ea013e78eab6916471a939178b648eacb4c9d68892d5fb1baf
3
+ size 4831938552
model-00042-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c4c318d084ffbce25446ead46fcfdcb04f2bbc94dd8a3e931e7b38e85b9e69ae
3
+ size 4907411096
model-00043-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:91a85d271e49df88616564a720602d36021c40b9487af708d0464c5054a858b5
3
+ size 4806747904
model-00044-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9f3fc049e4a652bde6e821fa6a102041865dd38b9e44f78cd0c54d4341d0185d
3
+ size 4831938544
model-00045-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:306ee61da392771772ba99c5c5f361d98db36a0f1bc87165792331d726a72a22
3
+ size 4831938552
model-00046-of-00051.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8b1fb1baeaaf87f0c7dc1e39e4337c9a00aef2e97f18e34b1020c76fc29f902f
3
+ size 4907411088