Junhoee commited on
Commit
4a8bf48
·
verified ·
1 Parent(s): d61e50e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +16 -6
README.md CHANGED
@@ -2,23 +2,33 @@
2
  base_model: Qwen/Qwen2.5-7B-Instruct
3
  library_name: peft
4
  pipeline_tag: text-generation
 
 
 
 
 
5
  ---
6
 
7
  # Model Card for Model ID
8
 
9
- <!-- Provide a quick summary of what the model is/does. -->
10
-
11
-
12
 
13
  ## Model Details
14
 
15
  ### Model Description
16
 
17
- <!-- Provide a longer summary of what this model is. -->
18
-
 
19
 
 
 
 
 
 
20
 
21
- - **Developed by:** [More Information Needed]
22
  - **Funded by [optional]:** [More Information Needed]
23
  - **Shared by [optional]:** [More Information Needed]
24
  - **Model type:** [More Information Needed]
 
2
  base_model: Qwen/Qwen2.5-7B-Instruct
3
  library_name: peft
4
  pipeline_tag: text-generation
5
+ language:
6
+ - en
7
+ tags:
8
+ - persona
9
+ - llm
10
  ---
11
 
12
  # Model Card for Model ID
13
 
14
+ I developed a persona LLM, also known as a role-play LLM.
15
+ The character is modeled after Megumin, a character from the novel Blessing of this Wonderful World.
 
16
 
17
  ## Model Details
18
 
19
  ### Model Description
20
 
21
+ This model is fine-tuned using the Qwen/Qwen2.5-7B-Instruct model as a mother model.
22
+ Due to the lack of GPU memory and resources, we used the QLoRA method to train only certain layer parts.
23
+ The learning factors were as follows
24
 
25
+ - learning_rate=5e-5
26
+ - lr_scheduler_type="cosine"
27
+ - warmup_steps=800
28
+ - num_train_epochs=5
29
+ - per_device_train_batch_size=8
30
 
31
+ - **Developed by:** [Junhoee Ku](https://github.com/junhoeKu)
32
  - **Funded by [optional]:** [More Information Needed]
33
  - **Shared by [optional]:** [More Information Needed]
34
  - **Model type:** [More Information Needed]