File size: 1,422 Bytes
3cc76bd 47d9603 bc50dc3 60b5d45 3cc76bd bc50dc3 3cc76bd |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
---
base_model:
- NousResearch/Llama-2-7b-hf
pipeline_tag: text-generation
library_name: peft
license: apache-2.0
datasets:
- mlabonne/guanaco-llama2-1k
metrics:
- bleu
- accuracy
language:
- en
tags:
- 'nlp '
- genetation
---
## Abstract
Llama 2 is a state-of-the-art large language model (LLM) developed by Meta AI, designed for a variety of natural language processing (NLP) tasks. Its architecture builds upon transformer-based models, leveraging massive text corpora during pretraining to develop rich language understanding capabilities. Fine-tuning Llama 2 can be customized to a specific task by using a smaller, task-specific dataset, often resulting in a specialized model that outperforms the general-purpose base model on that task. In this project, we fine-tune the Llama-2-7b-hf model using a subset of the Guanaco dataset, focusing on developing a highly efficient model called "MiniGuanaco."
## Project Overview
This repository demonstrates the steps and code required to fine-tune Llama 2 for specific tasks. Using the Hugging Face model **NousResearch/Llama-2-7b-hf** as the base, the model is fine-tuned with the dataset **mlabonne/guanaco-llama2-1k**. The fine-tuned model is saved under the name **llama2-miniguanaco**.
# You can access more information through:
- [Github](https://github.com/zeyadusf/FineTune-Llama2)
- [Kaggle](https://www.kaggle.com/code/zeyadusf/finetune-llama2/notebook) |