Upload README.md
Browse files
README.md
CHANGED
|
@@ -25,6 +25,13 @@ It is based on the Guanaco LORA of LLaMA weighing in at 65B parameters.
|
|
| 25 |
The primary goal of this model is to improve question-answering and medical dialogue tasks.
|
| 26 |
It was trained using [LoRA](https://arxiv.org/abs/2106.09685) and quantized, to reduce memory footprint.
|
| 27 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 28 |
|
| 29 |
> The following README is taken from the source page [medalpaca](https://huggingface.co/medalpaca/medalpaca-lora-13b-8bit)
|
| 30 |
|
|
|
|
| 25 |
The primary goal of this model is to improve question-answering and medical dialogue tasks.
|
| 26 |
It was trained using [LoRA](https://arxiv.org/abs/2106.09685) and quantized, to reduce memory footprint.
|
| 27 |
|
| 28 |
+
Steps to load this model:
|
| 29 |
+
1. Load Guanaco-65-GPTQ https://huggingface.co/TheBloke/guanaco-65B-GPTQ
|
| 30 |
+
* I recommend using text-generation-ui to test it out: https://github.com/oobabooga/text-generation-webui/tree/main
|
| 31 |
+
2. Download this LoRA and apply it to the model
|
| 32 |
+
3. ensure `--monkey-path` is enabled in the text-generation-ui, 4-bit instructions [here](https://github.com/oobabooga/text-generation-webui/blob/main/docs/GPTQ-models-(4-bit-mode).md)
|
| 33 |
+
|
| 34 |
+
---
|
| 35 |
|
| 36 |
> The following README is taken from the source page [medalpaca](https://huggingface.co/medalpaca/medalpaca-lora-13b-8bit)
|
| 37 |
|