Text Generation
Transformers
GGUF
llama
Inference Endpoints
YanaS commited on
Commit
e95191e
1 Parent(s): 1643bbf

Upload quantized model versions of Photolens/llama-2-7b-langchain-chat

Browse files
.gitattributes CHANGED
@@ -33,3 +33,7 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ llama-2-7b-langchain-chat-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
37
+ llama-2-7b-langchain-chat-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
38
+ llama-2-7b-langchain-chat-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
39
+ llama-2-7b-langchain-chat-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,76 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ - es
5
+ - ru
6
+ - de
7
+ - pl
8
+ - th
9
+ - vi
10
+ - sv
11
+ - bn
12
+ - da
13
+ - he
14
+ - it
15
+ - fa
16
+ - sk
17
+ - id
18
+ - nb
19
+ - el
20
+ - nl
21
+ - hu
22
+ - eu
23
+ - zh
24
+ - eo
25
+ - ja
26
+ - ca
27
+ - cs
28
+ - bg
29
+ - fi
30
+ - pt
31
+ - tr
32
+ - ro
33
+ - ar
34
+ - uk
35
+ - gl
36
+ - fr
37
+ - ko
38
+ task_categories:
39
+ - conversational
40
+ license: llama2
41
+ datasets:
42
+ - Photolens/oasst1-langchain-llama-2-formatted
43
+ ---
44
+
45
+ ## Model Overview
46
+ Model license: Llama-2<br>
47
+ This model is trained based on [NousResearch/Llama-2-7b-chat-hf](https://huggingface.co/NousResearch/Llama-2-7b-chat-hf) model that is QLoRA finetuned on [Photolens/oasst1-langchain-llama-2-formatted](https://huggingface.co/datasets/Photolens/oasst1-langchain-llama-2-formatted) dataset.<br>
48
+
49
+ ## Prompt Template: Llama-2
50
+ ```
51
+ <s>[INST] Prompter Message [/INST] Assistant Message </s>
52
+ ```
53
+
54
+ ## Intended Use
55
+ Dataset that is used to finetune base model is optimized for langchain applications.<br>
56
+ So this model is intended for a langchain LLM.
57
+
58
+ ## Training Details
59
+ This model took `1:14:16` to train in QLoRA on a single `A100 40gb` GPU.<br>
60
+ - *epochs*: `1`
61
+ - *train batch size*: `8`
62
+ - *eval batch size*: `8`
63
+ - *gradient accumulation steps*: `1`
64
+ - *maximum gradient normal*: `0.3`
65
+ - *learning rate*: `2e-4`
66
+ - *weight decay*: `0.001`
67
+ - *optimizer*: `paged_adamw_32bit`
68
+ - *learning rate schedule*: `cosine`
69
+ - *warmup ratio (linear)*: `0.03`
70
+
71
+ ## Models in this series
72
+ | Model | Train time | Size (in params) | Base Model |
73
+ ---|---|---|---
74
+ | [llama-2-7b-langchain-chat](https://huggingface.co/Photolens/llama-2-7b-langchain-chat/) | 1:14:16 | 7 billion | [NousResearch/Llama-2-7b-chat-hf](https://huggingface.co/NousResearch/Llama-2-7b-chat-hf) |
75
+ | [llama-2-13b-langchain-chat](https://huggingface.co/Photolens/llama-2-13b-langchain-chat/) | 2:50:27 | 13 billion | [TheBloke/Llama-2-13B-Chat-fp16](https://huggingface.co/TheBloke/Llama-2-13B-Chat-fp16) |
76
+ | [Photolens/OpenOrcaxOpenChat-2-13b-langchain-chat](https://huggingface.co/Photolens/OpenOrcaxOpenChat-2-13b-langchain-chat/) | 2:56:54 | 13 billion | [Open-Orca/OpenOrcaxOpenChat-Preview2-13B](https://huggingface.co/Open-Orca/OpenOrcaxOpenChat-Preview2-13B) |
config.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "NousResearch/Llama-2-7b-chat-hf",
3
+ "architectures": [
4
+ "LlamaForCausalLM"
5
+ ],
6
+ "bos_token_id": 1,
7
+ "eos_token_id": 2,
8
+ "hidden_act": "silu",
9
+ "hidden_size": 4096,
10
+ "initializer_range": 0.02,
11
+ "intermediate_size": 11008,
12
+ "max_position_embeddings": 4096,
13
+ "model_type": "llama",
14
+ "num_attention_heads": 32,
15
+ "num_hidden_layers": 32,
16
+ "num_key_value_heads": 32,
17
+ "pad_token_id": 0,
18
+ "pretraining_tp": 1,
19
+ "rms_norm_eps": 1e-05,
20
+ "rope_scaling": null,
21
+ "tie_word_embeddings": false,
22
+ "torch_dtype": "float16",
23
+ "transformers_version": "4.31.0",
24
+ "use_cache": true,
25
+ "vocab_size": 32000
26
+ }
llama-2-7b-langchain-chat-Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:750e96efc7b6220462eabd02938ba1410c991773896acca1317e6da51b623a3e
3
+ size 4081004224
llama-2-7b-langchain-chat-Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:00c1c6129bccc237cdb85b065a25b8826f96464c94946a4f28a11313f10f0362
3
+ size 4783156928
llama-2-7b-langchain-chat-Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9a625393ec6f72b60c3c0c7cd7db547b875c9becb6a8d43288036c73f8c4f7b7
3
+ size 5529194176
llama-2-7b-langchain-chat-Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:84c321bc9173ded1e14d5a4139f524c4f809eb06c9c5ebe68a8dfd4deb43dc27
3
+ size 7161089728