|
--- |
|
base_model: |
|
- NousResearch/Llama-2-7b-hf |
|
pipeline_tag: text-generation |
|
library_name: peft |
|
license: apache-2.0 |
|
datasets: |
|
- mlabonne/guanaco-llama2-1k |
|
metrics: |
|
- bleu |
|
- accuracy |
|
language: |
|
- en |
|
tags: |
|
- 'nlp ' |
|
- genetation |
|
--- |
|
## Abstract |
|
|
|
Llama 2 is a state-of-the-art large language model (LLM) developed by Meta AI, designed for a variety of natural language processing (NLP) tasks. Its architecture builds upon transformer-based models, leveraging massive text corpora during pretraining to develop rich language understanding capabilities. Fine-tuning Llama 2 can be customized to a specific task by using a smaller, task-specific dataset, often resulting in a specialized model that outperforms the general-purpose base model on that task. In this project, we fine-tune the Llama-2-7b-hf model using a subset of the Guanaco dataset, focusing on developing a highly efficient model called "MiniGuanaco." |
|
|
|
## Project Overview |
|
|
|
This repository demonstrates the steps and code required to fine-tune Llama 2 for specific tasks. Using the Hugging Face model **NousResearch/Llama-2-7b-hf** as the base, the model is fine-tuned with the dataset **mlabonne/guanaco-llama2-1k**. The fine-tuned model is saved under the name **llama2-miniguanaco**. |
|
|
|
# You can access more information through: |
|
- [Github](https://github.com/zeyadusf/FineTune-Llama2) |
|
- [Kaggle](https://www.kaggle.com/code/zeyadusf/finetune-llama2/notebook) |
