Update README.md

README.md

@@ -7,17 +7,64 @@ tags:
 - text-generation-inference
 - transformers
 - unsloth
-- gemma2
+- llama
 - trl
 - sft
 ---
 
-# Uploaded  model
+# MedGPT-Gemma2-9B-v.1
 
-- **Developed by:** valeriojob
-- **License:** apache-2.0
-- **Finetuned from model :** unsloth/gemma-2-9b
+- This model is a fine-tuned version of [unsloth/gemma-2-9b](https://huggingface.co/unsloth/gemma-2-9b) on an dataset created by [Valerio Job](https://huggingface.co/valeriojob) together with GPs based on real medical data.
+- Version 1 (v.1) of MedGPT is the very first version of MedGPT and the training dataset has been kept simple and small with only 60 examples.
+- This repo includes the 16bit format of the model as well as the LoRA adapters of the model. There is a separate repo called [valeriojob/MedGPT-Gemma2-9B-BA-v.1-GGUF](https://huggingface.co/valeriojob/MedGPT-Gemma2-9B-BA-v.1-GGUF) that includes the quantized versions of this model in GGUF format.
+- This model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
 
-This gemma2 model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
+## Model description
 
-[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
+This model acts as a supplementary assistance to GPs helping them in medical and admin tasks.
+
+## Intended uses & limitations
+
+The fine-tuned model should not be used in production! This model has been created as a initial prototype in the context of a bachelor thesis.
+
+## Training and evaluation data
+
+The dataset (train and test) used for fine-tuning this model can be found here: [datasets/valeriojob/BA-v.1](https://huggingface.co/datasets/valeriojob/BA-v.1)
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- per_device_train_batch_size = 2,
+- gradient_accumulation_steps = 4,
+- warmup_steps = 5,
+- max_steps = 60,
+- learning_rate = 2e-4,
+- fp16 = not is_bfloat16_supported(),
+- bf16 = is_bfloat16_supported(),
+- logging_steps = 1,
+- optim = "adamw_8bit",
+- weight_decay = 0.01,
+- lr_scheduler_type = "linear",
+- seed = 3407,
+- output_dir = "outputs"
+
+### Training results
+
+| Training Loss | Step |
+|:-------------:|:----:|
+| 2.237900      | 1    |
+| 2.292200      | 2    |
+| 2.215200      | 3    |
+| 1.561200      | 5    |
+| 0.584500      | 10   |
+| 0.372500      | 15   |
+| 0.258600      | 20   |
+| 0.126300      | 30   |
+| 0.064100      | 40   |
+| 0.040800      | 50   |
+| 0.045700      | 60   |
+
+## Licenses
+- **License:** apache-2.0