Text Generation
Transformers
Safetensors
English
Japanese
gemma2
text-generation-inference
unsloth
trl
conversational
Inference Endpoints
bay-llm commited on
Commit
e29044e
·
verified ·
1 Parent(s): 8767580

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +87 -4
README.md CHANGED
@@ -1,22 +1,105 @@
1
  ---
2
- base_model: unsloth/gemma-2-9b-bnb-4bit
 
3
  tags:
4
  - text-generation-inference
5
  - transformers
6
  - unsloth
7
  - gemma2
8
  - trl
9
- license: apache-2.0
10
  language:
11
  - en
 
 
 
 
 
12
  ---
13
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
14
  # Uploaded model
15
 
16
  - **Developed by:** bay-llm
17
- - **License:** apache-2.0
18
  - **Finetuned from model :** unsloth/gemma-2-9b-bnb-4bit
19
 
20
  This gemma2 model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
21
 
22
- [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
 
1
  ---
2
+ base_model:
3
+ - google/gemma-2-9b
4
  tags:
5
  - text-generation-inference
6
  - transformers
7
  - unsloth
8
  - gemma2
9
  - trl
10
+ license: gemma
11
  language:
12
  - en
13
+ - ja
14
+ datasets:
15
+ - kanhatakeyama/wizardlm8x22b-logical-math-coding-sft_additional-ja
16
+ - kanhatakeyama/AutoMultiTurnByCalm3-22B
17
+ - kanhatakeyama/ramdom-to-fixed-multiturn-Calm3
18
  ---
19
 
20
+ # Model Card for Model ID
21
+ Instruction tuning
22
+ The models have been fine-tuned.
23
+
24
+ Usage
25
+ ```python
26
+ !pip install vllm==0.6.4.post1 --force-reinstall
27
+
28
+ import time
29
+ import torch
30
+ import transformers
31
+ from transformers import (
32
+ AutoTokenizer,
33
+ AutoModelForCausalLM,
34
+ )
35
+ import vllm ### packaging==24.1にしないとエラーになる!! ###
36
+ print(vllm.__version__)
37
+
38
+ MAX_LENGTH = 1000
39
+ MODEL_NAME = "bay-llm/gemma-9b-SFT-180-16bit" # コンペで提出したいモデルに適宜置換
40
+
41
+ llm = vllm.LLM(
42
+ model=MODEL_NAME,
43
+ tensor_parallel_size=1,
44
+ gpu_memory_utilization=0.95,
45
+ trust_remote_code=True,
46
+ max_model_len=1024,
47
+
48
+ )
49
+ tokenizer = llm.get_tokenizer()
50
+
51
+ # ELYZA-tasks-100-TVの読み込み。事前にファイルをアップロードしてください
52
+ # データセットの読み込み。
53
+ # omnicampusの開発環境では、左にタスクのjsonlをドラッグアンドドロップしてから実行。
54
+ import json
55
+ datasets = []
56
+ with open("../elyza-tasks-100-TV_0.jsonl", "r") as f:
57
+ item = ""
58
+ for line in f:
59
+ line = line.strip()
60
+ item += line
61
+ if item.endswith("}"):
62
+ datasets.append(json.loads(item))
63
+ item = ""
64
+
65
+ print(datasets[0])
66
+
67
+ messages_list = [
68
+ [{"role": "user", "content": datasets[i]["input"]}] for i in range(len(datasets))
69
+ ]
70
+
71
+ prompts = [line[0]["content"] for line in messages_list]
72
+ prompt_token_ids = [tokenizer.apply_chat_template(messages, add_generation_prompt=True) for messages in messages_list]
73
+ sampling_params = vllm.SamplingParams(
74
+ temperature=0.5,
75
+ max_tokens=512,
76
+ )
77
+ outputs = llm.generate(prompt_token_ids=prompt_token_ids, sampling_params=sampling_params)
78
+ for prompt, response in zip(prompts, outputs):
79
+ print("prompt:", prompt)
80
+ print("output:", response.outputs[0].text.strip())
81
+ print("-"*80)
82
+
83
+ import json
84
+ data = [{
85
+ "task_id": i,
86
+ "input": prompts[i],
87
+ "output": outputs[i].outputs[0].text.strip()
88
+ } for i in range(len(datasets))]
89
+ file_path = 'submmit.jsonl'
90
+ with open(file_path, 'w', encoding='utf-8') as file:
91
+ for entry in data:
92
+ json.dump(entry, file, ensure_ascii=False)
93
+ file.write('\n')
94
+
95
+ ```
96
+
97
  # Uploaded model
98
 
99
  - **Developed by:** bay-llm
100
+ - **License:** gemma
101
  - **Finetuned from model :** unsloth/gemma-2-9b-bnb-4bit
102
 
103
  This gemma2 model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
104
 
105
+ [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)