README.md

---
license: cc-by-nc-4.0
base_model: KT-AI/midm-bitext-S-7B-inst-v1
tags:
- generated_from_trainer
model-index:
- name: lora-midm-7b-nsmc-review-understanding
  results: []
datasets:
- nsmc
---

# lora-midm-7b-nsmc-review-understanding

This model is a fine-tuned version of [KT-AI/midm-bitext-S-7B-inst-v1](https://huggingface.co./KT-AI/midm-bitext-S-7B-inst-v1) on an unknown dataset.

## Model description

nsmc data 기반 미세튜닝 모델

## Intended uses & limitations

More information needed

## Training and evaluation data

training data로 nsmc train data 앞쪽 2000개,
evaluation data로 nsmc test data 앞쪽 1000개를 사용했습니다.

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 1
- eval_batch_size: 1
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 2
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.03
- training_steps: 200
- mixed_precision_training: Native AMP

### Training results

총 200step 돌렸습니다. 50step마다 check한 결과는 아래와 같습니다.  
50 step training loss: 1.6881  
100 step training loss: 1.1443  
150 step training loss: 1.0563  
200 step training loss: 1.0446  

## 실험 내용 및 분류 결과
미세튜닝한 모델에 nsmc test data 1000개를 입력으로 주어 긍정 또는 부정 단어를 생성하도록 했습니다.  
단어 생성 결과 '긍정' 444개, '부정' 532개, ',' 4개, '정' 20개 입니다.  
정확도는 정답수 / 1000 * 100으로 계산했으며, 결과는 87.80% 입니다.  

### Framework versions

- Transformers 4.35.2
- Pytorch 2.1.0+cu118
- Datasets 2.15.0
- Tokenizers 0.15.0