hzhwcmhf commited on
Commit
c096839
·
verified ·
1 Parent(s): d76cadc

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -6
README.md CHANGED
@@ -68,10 +68,6 @@ text = tokenizer.apply_chat_template(
68
  add_generation_prompt=True
69
  )
70
 
71
- # avoid empty thought content by forcing the model to start with "<think>\n"
72
- response_prefix = "<think>\n"
73
- text += response_prefix
74
-
75
  model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
76
 
77
  generated_ids = model.generate(
@@ -83,14 +79,14 @@ generated_ids = [
83
  ]
84
 
85
  response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
86
- print(response_prefix + response)
87
  ```
88
 
89
  ### Usage Guidelines
90
 
91
  To achieve optimal performance, we recommend the following settings:
92
 
93
- 1. **Enforce Thoughtful Output**: Ensure the model starts with "\<think\>\n" to prevent generating empty thinking content, which can degrade output quality.
94
 
95
  2. **Sampling Parameters**:
96
  - Use Temperature=0.6 and TopP=0.95 instead of Greedy decoding to avoid endless repetitions and enhance diversity.
 
68
  add_generation_prompt=True
69
  )
70
 
 
 
 
 
71
  model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
72
 
73
  generated_ids = model.generate(
 
79
  ]
80
 
81
  response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
82
+ print(response)
83
  ```
84
 
85
  ### Usage Guidelines
86
 
87
  To achieve optimal performance, we recommend the following settings:
88
 
89
+ 1. **Enforce Thoughtful Output**: Ensure the model starts with "\<think\>\n" to prevent generating empty thinking content, which can degrade output quality. If you use `apply_chat_template` and set `add_generation_prompt=True`, this is already automatically implemented, but it may cause the response to lack the \<think\> tag at the beginning. This is normal behavior.
90
 
91
  2. **Sampling Parameters**:
92
  - Use Temperature=0.6 and TopP=0.95 instead of Greedy decoding to avoid endless repetitions and enhance diversity.