Create README.md

README.md

@@ -0,0 +1,74 @@
+---
+title: Yo!Medical3000
+tags:
+  - healthcare
+  - NLP
+  - dialogues
+  - LLM
+  - fine-tuned
+license: unknown
+datasets:
+  - Kabatubare/medical-guanaco-3000
+---
+
+# Yo!Medical3000 Model Card
+
+This is a model card for Yo!Medical3000, a fine-tuned version of TinyPixel/Llama-2-7B-bf16-sharded, specifically aimed at medical dialogues.
+
+## Model Details
+
+### Base Model
+
+- **Name**: TinyPixel/Llama-2-7B-bf16-sharded
+- **Description**: (A brief description of the base model, its architecture, and its intended use-cases)
+  
+### Fine-tuned Model
+
+- **Name**: Yo!Medical3000
+- **Fine-tuned on**: Kabatubare/medical-guanaco-3000
+- **Description**: This model is fine-tuned to specialize in medical dialogues and healthcare applications.
+
+### Architecture and Training Parameters
+
+#### Architecture
+
+- **LoRA Attention Dimension**: 64
+- **LoRA Alpha Parameter**: 16
+- **LoRA Dropout**: 0.1
+- **Precision**: 4-bit (bitsandbytes)
+- **Quantization Type**: nf4
+
+#### Training Parameters
+
+- **Epochs**: 3
+- **Batch Size**: 4
+- **Gradient Accumulation Steps**: 1
+- **Max Gradient Norm**: 0.3
+- **Learning Rate**: 3e-4
+- **Weight Decay**: 0.001
+- **Optimizer**: paged_adamw_32bit
+- **LR Scheduler**: cosine
+- **Warmup Ratio**: 0.03
+- **Logging Steps**: 25
+
+## Datasets
+
+### Base Model Dataset
+
+- **Name**: (Name of the dataset used for the base model)
+- **Description**: (A brief description of this dataset and its characteristics)
+
+### Fine-tuning Dataset
+
+- **Name**: Kabatubare/medical-guanaco-3000
+- **Description**: This is a reduced and balanced dataset curated from a larger medical dialogue dataset. It aims to cover a broad range of medical topics and is suitable for training healthcare chatbots and conducting medical NLP research.
+
+## Usage
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+tokenizer = AutoTokenizer.from_pretrained("Yo!Medical3000")
+model = AutoModelForCausalLM.from_pretrained("Yo!Medical3000")
+
+# Use the model for inference