license: apache-2.0
library_name: adapter-transformers
Model Card for Mistral-7B-Instruct-v0.1-MyRestaurant-Domain-Adaptation
This is a domain adaptation for questions about My Restaurant
You can play by asking the model questions about the menu...
Foundation Model : https://huggingface.co./mistralai/Mistral-7B-Instruct-v0.1
Dataset : https://huggingface.co./datasets/Argen7um/restrant-qa
The model has been fine tuned with 2 x GPU T4 (RAM : 2 x 14.8GB) + CPU (RAM : 29GB).
The model is based upon the foundation model : Mistral-7B.
It has been tuned with Supervised Fine-tuning Trainer and PEFT LoRa.
Notebook used for the training
You can find it in the files and versions tab
Direct link : https://huggingface.co./Laurent1/Mistral-7B-Instruct-v0.1-MyRestaurant-Domain-Adaptation/blob/main/laurent-restaurant-adaptation-mistral-7b-tuned.ipynb
Bias, Risks, and Limitations
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model.
Generation of plausible yet incorrect factual information, termed hallucination, is an unsolved issue in large language models.
Training Details
- per_device_train_batch_size = 1
- gradient_accumulation_steps = 16
- 2 x GPU T4 (RAM : 14.8GB) + CPU (RAM : 29GB)