Update README.md
Browse files
README.md
CHANGED
@@ -68,10 +68,6 @@ text = tokenizer.apply_chat_template(
|
|
68 |
add_generation_prompt=True
|
69 |
)
|
70 |
|
71 |
-
# avoid empty thought content by forcing the model to start with "<think>\n"
|
72 |
-
response_prefix = "<think>\n"
|
73 |
-
text += response_prefix
|
74 |
-
|
75 |
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
|
76 |
|
77 |
generated_ids = model.generate(
|
@@ -83,14 +79,14 @@ generated_ids = [
|
|
83 |
]
|
84 |
|
85 |
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
|
86 |
-
print(
|
87 |
```
|
88 |
|
89 |
### Usage Guidelines
|
90 |
|
91 |
To achieve optimal performance, we recommend the following settings:
|
92 |
|
93 |
-
1. **Enforce Thoughtful Output**: Ensure the model starts with "\<think\>\n" to prevent generating empty thinking content, which can degrade output quality.
|
94 |
|
95 |
2. **Sampling Parameters**:
|
96 |
- Use Temperature=0.6 and TopP=0.95 instead of Greedy decoding to avoid endless repetitions and enhance diversity.
|
|
|
68 |
add_generation_prompt=True
|
69 |
)
|
70 |
|
|
|
|
|
|
|
|
|
71 |
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
|
72 |
|
73 |
generated_ids = model.generate(
|
|
|
79 |
]
|
80 |
|
81 |
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
|
82 |
+
print(response)
|
83 |
```
|
84 |
|
85 |
### Usage Guidelines
|
86 |
|
87 |
To achieve optimal performance, we recommend the following settings:
|
88 |
|
89 |
+
1. **Enforce Thoughtful Output**: Ensure the model starts with "\<think\>\n" to prevent generating empty thinking content, which can degrade output quality. If you use `apply_chat_template` and set `add_generation_prompt=True`, this is already automatically implemented, but it may cause the response to lack the \<think\> tag at the beginning. This is normal behavior.
|
90 |
|
91 |
2. **Sampling Parameters**:
|
92 |
- Use Temperature=0.6 and TopP=0.95 instead of Greedy decoding to avoid endless repetitions and enhance diversity.
|