mattshumer commited on
Commit
688a998
1 Parent(s): 7a2427f

Upload folder using huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +56 -0
README.md ADDED
@@ -0,0 +1,56 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - meta-llama/Meta-Llama-3.1-8B-Instruct
4
+ license: apache-2.0
5
+ tags:
6
+ - merge
7
+ - mergekit
8
+ - lazymergekit
9
+ - meta-llama/Meta-Llama-3.1-8B-Instruct
10
+ ---
11
+
12
+ # 70b-ft-plus-instruct-dareties
13
+
14
+ 70b-ft-plus-instruct-dareties is a merge of the following models using [LazyMergekit](https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb?usp=sharing):
15
+ * [meta-llama/Meta-Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct)
16
+
17
+ ## 🧩 Configuration
18
+
19
+ ```yaml
20
+ models:
21
+ - model: mattshumer/llama3.1-test-7k-8b-1
22
+ - model: meta-llama/Meta-Llama-3.1-8B-Instruct
23
+ parameters:
24
+ density: 0.53
25
+ weight: 0.6
26
+ merge_method: dare_ties
27
+ base_model: meta-llama/Meta-Llama-3.1-8B-Instruct
28
+ parameters:
29
+ int8_mask: true
30
+ dtype: bfloat16
31
+ ```
32
+
33
+ ## 💻 Usage
34
+
35
+ ```python
36
+ !pip install -qU transformers accelerate
37
+
38
+ from transformers import AutoTokenizer
39
+ import transformers
40
+ import torch
41
+
42
+ model = "mattshumer/70b-ft-plus-instruct-dareties"
43
+ messages = [{"role": "user", "content": "What is a large language model?"}]
44
+
45
+ tokenizer = AutoTokenizer.from_pretrained(model)
46
+ prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
47
+ pipeline = transformers.pipeline(
48
+ "text-generation",
49
+ model=model,
50
+ torch_dtype=torch.float16,
51
+ device_map="auto",
52
+ )
53
+
54
+ outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
55
+ print(outputs[0]["generated_text"])
56
+ ```