Llama-2-7b-chat-Calgary-Cambridge-multi-turn
NousResearch/Llama-2-7b-chat-hf fine-tuned on kazcfz/Medical-Calgary-Cambridge-multi-turn-llama2-37
QLoRA parameters
| Parameter | Value |
|---|---|
| lora_r | 64 |
| lora_alpha | 16 |
| lora_dropout | 0.1 |
bitsandbytes parameters
| Parameter | Value |
|---|---|
| use_4bit | True |
| bnb_4bit_compute_dtype | "float16" |
| bnb_4bit_quant_type | "nf4" |
| use_nested_quant | False |
TrainingArguments parameters
| Parameter | Value |
|---|---|
| num_train_epochs | 10 |
| fp16 | False |
| bf16 | False |
| per_device_train_batch_size | 4 |
| gradient_accumulation_steps | 1 |
| gradient_checkpointing | True |
| max_grad_norm | 0.3 |
| learning_rate | 2e-4 |
| weight_decay | 0.001 |
| optim | "paged_adamw_32bit" |
| lr_scheduler_type | "cosine" |
| max_steps | -1 |
| warmup_ratio | 0.03 |
| group_by_length | True |
| save_steps | 0 |
| logging_steps | 25 |
SFT parameters
| Parameter | Value |
|---|---|
| max_seq_length | None |
| packing | False |
Prompt template: Llama-2-Chat
<s>[INST] {prompt} [/INST]
Sample (actual) output (Max length: 200)
Base Model
Query: <s>[INST] What is the Golaif Virus? [/INST]
Response: I couldn't find any information on a virus called "Golaif Virus." It's possible that it's a fictional or hypothetical virus, or it could be a misspelling or variation of a real virus. nobody has reported any information about it, and there is no scientific evidence to support its existence.
If you have any more information or context about the Golaif Virus, please let me know and I'll do my best to help you.
Fine-tuned
Query: <s>[INST] What is the Golaif Virus? [/INST]
Response: I apologize, but there is no such thing as the "Golaif Virus." It'<s a fictional term that doesn't exist in medical literature. nobody has ever reported a virus by that name. It's essential to rely on credible sources when discussing health topics, and I'm here to help you with any questions you may have. If you have any specific questions or concerns about viruses or other health-related topics, feel free to ask!
Reload model + Merge LoRA weights?
Query: <s>[INST] What is the Golaif Virus? [/INST]
Response: I apologize, but there is no known virus called "Golaif Virus." It is possible that you may have misspelled the name or that it is a fictional term. Additionally, I couldn't find any information on a virus with this name through reputable medical sources. If you have any specific questions or concerns about a particular virus, feel free to ask, and I'll do my best to help.