README.md

---
base_model: meta-llama/Llama-2-7b-chat-hf
tags:
- generated_from_trainer
model-index:
- name: lora-llama-2-7b-nsmc-review-understanding
  results: []
datasets:
- nsmc
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# lora-llama-2-7b-nsmc-review-understanding

This model is a fine-tuned version of [meta-llama/Llama-2-7b-chat-hf](https://huggingface.co./meta-llama/Llama-2-7b-chat-hf) on an unknown dataset.

## Model description

nsmc data 기반 미세튜닝 모델

## Intended uses & limitations

More information needed

## Training and evaluation data

training data로 nsmc train data 앞쪽 2000개, evaluation data로 nsmc test data 앞쪽 1000개를 사용했습니다.

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 1
- eval_batch_size: 1
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 2
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.03
- training_steps: 200
- mixed_precision_training: Native AMP

### Training results

총 200step 돌렸습니다. 50step마다 check한 결과는 아래와 같습니다.  
50 step training loss: 1.2201  
100 step training loss: 0.8892  
150 step training loss: 0.8449  
200 step training loss: 0.8370  

## 실험 내용 및 분류 결과

미세튜닝한 모델에 nsmc test data 1000개를 입력으로 주어 긍정 또는 부정 단어를 생성하도록 했습니다.  
단어 생성 결과는 '긍정' 443개, '부정' 556개, '부산엔 2015년 12월 17일 개봉했습니다. ###Midm;부정' 1개 입니다.  
정확도는 정답수 / 1000 * 100으로 계산했으며, 결과는 84.90% 입니다.

### Framework versions

- Transformers 4.35.2
- Pytorch 2.1.0+cu118
- Datasets 2.15.0
- Tokenizers 0.15.0