--- license: mit library_name: peft tags: - trl - reward-trainer - generated_from_trainer base_model: openai-community/gpt2-large metrics: - accuracy model-index: - name: RM-HH-GPT2-4w_helpful_gpt3_loraR64_40000_gpt2-large_shuffleTrue_extractchosenFalse results: [] --- # RM-HH-GPT2-4w_helpful_gpt3_loraR64_40000_gpt2-large_shuffleTrue_extractchosenFalse This model is a fine-tuned version of [openai-community/gpt2-large](https://huggingface.co./openai-community/gpt2-large) on the None dataset. It achieves the following results on the evaluation set: - Loss: 0.4920 - Accuracy: 0.7471 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 1.41e-05 - train_batch_size: 1 - eval_batch_size: 8 - seed: 42 - gradient_accumulation_steps: 4 - total_train_batch_size: 4 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - num_epochs: 2.0 ### Training results | Training Loss | Epoch | Step | Validation Loss | Accuracy | |:-------------:|:-----:|:-----:|:---------------:|:--------:| | 0.6411 | 0.02 | 250 | 0.6110 | 0.6618 | | 0.5909 | 0.04 | 500 | 0.5606 | 0.6981 | | 0.5266 | 0.06 | 750 | 0.5351 | 0.7134 | | 0.5588 | 0.08 | 1000 | 0.5273 | 0.7154 | | 0.5328 | 0.1 | 1250 | 0.5219 | 0.7194 | | 0.5059 | 0.13 | 1500 | 0.5198 | 0.7252 | | 0.5096 | 0.15 | 1750 | 0.5158 | 0.7264 | | 0.5212 | 0.17 | 2000 | 0.5152 | 0.7209 | | 0.511 | 0.19 | 2250 | 0.5130 | 0.7232 | | 0.5286 | 0.21 | 2500 | 0.5098 | 0.7237 | | 0.5147 | 0.23 | 2750 | 0.5076 | 0.7267 | | 0.4938 | 0.25 | 3000 | 0.5068 | 0.7277 | | 0.5279 | 0.27 | 3250 | 0.5040 | 0.7290 | | 0.5003 | 0.29 | 3500 | 0.5050 | 0.7303 | | 0.5069 | 0.31 | 3750 | 0.5006 | 0.7298 | | 0.4992 | 0.33 | 4000 | 0.4987 | 0.7298 | | 0.4925 | 0.36 | 4250 | 0.4989 | 0.7347 | | 0.4984 | 0.38 | 4500 | 0.4973 | 0.7324 | | 0.4995 | 0.4 | 4750 | 0.4956 | 0.7335 | | 0.5038 | 0.42 | 5000 | 0.4937 | 0.7341 | | 0.4892 | 0.44 | 5250 | 0.4945 | 0.7371 | | 0.5017 | 0.46 | 5500 | 0.4942 | 0.7377 | | 0.5105 | 0.48 | 5750 | 0.4947 | 0.7377 | | 0.4528 | 0.5 | 6000 | 0.4991 | 0.7367 | | 0.5055 | 0.52 | 6250 | 0.4999 | 0.7369 | | 0.492 | 0.54 | 6500 | 0.4954 | 0.7390 | | 0.4632 | 0.57 | 6750 | 0.4991 | 0.7416 | | 0.5061 | 0.59 | 7000 | 0.4926 | 0.7412 | | 0.4707 | 0.61 | 7250 | 0.4964 | 0.7426 | | 0.4728 | 0.63 | 7500 | 0.4948 | 0.7412 | | 0.4914 | 0.65 | 7750 | 0.4880 | 0.7437 | | 0.4754 | 0.67 | 8000 | 0.4859 | 0.7412 | | 0.4875 | 0.69 | 8250 | 0.4924 | 0.7433 | | 0.5225 | 0.71 | 8500 | 0.4854 | 0.7437 | | 0.5135 | 0.73 | 8750 | 0.4853 | 0.7443 | | 0.4589 | 0.75 | 9000 | 0.4921 | 0.7411 | | 0.4751 | 0.77 | 9250 | 0.4913 | 0.7412 | | 0.4848 | 0.8 | 9500 | 0.4858 | 0.7420 | | 0.5003 | 0.82 | 9750 | 0.4837 | 0.7429 | | 0.4636 | 0.84 | 10000 | 0.4894 | 0.7429 | | 0.493 | 0.86 | 10250 | 0.4845 | 0.7446 | | 0.4944 | 0.88 | 10500 | 0.4909 | 0.7424 | | 0.487 | 0.9 | 10750 | 0.4872 | 0.7444 | | 0.488 | 0.92 | 11000 | 0.4901 | 0.7422 | | 0.4622 | 0.94 | 11250 | 0.4863 | 0.7441 | | 0.4909 | 0.96 | 11500 | 0.4816 | 0.7433 | | 0.4626 | 0.98 | 11750 | 0.4910 | 0.7414 | | 0.4911 | 1.0 | 12000 | 0.4913 | 0.7420 | | 0.4674 | 1.03 | 12250 | 0.4961 | 0.7456 | | 0.4748 | 1.05 | 12500 | 0.4967 | 0.7448 | | 0.4693 | 1.07 | 12750 | 0.4975 | 0.7460 | | 0.4943 | 1.09 | 13000 | 0.4950 | 0.7454 | | 0.4912 | 1.11 | 13250 | 0.4958 | 0.7446 | | 0.4845 | 1.13 | 13500 | 0.4977 | 0.7435 | | 0.4906 | 1.15 | 13750 | 0.4983 | 0.7444 | | 0.4785 | 1.17 | 14000 | 0.4969 | 0.7439 | | 0.4546 | 1.19 | 14250 | 0.5068 | 0.7435 | | 0.4625 | 1.21 | 14500 | 0.5018 | 0.7435 | | 0.5086 | 1.23 | 14750 | 0.4996 | 0.7422 | | 0.4574 | 1.26 | 15000 | 0.5048 | 0.7444 | | 0.4655 | 1.28 | 15250 | 0.4930 | 0.7454 | | 0.4796 | 1.3 | 15500 | 0.4938 | 0.7446 | | 0.4924 | 1.32 | 15750 | 0.4830 | 0.7486 | | 0.4952 | 1.34 | 16000 | 0.4910 | 0.7471 | | 0.4298 | 1.36 | 16250 | 0.4939 | 0.7465 | | 0.4324 | 1.38 | 16500 | 0.5072 | 0.7458 | | 0.4831 | 1.4 | 16750 | 0.5112 | 0.7454 | | 0.5154 | 1.42 | 17000 | 0.5019 | 0.7444 | | 0.4629 | 1.44 | 17250 | 0.4982 | 0.7461 | | 0.5071 | 1.46 | 17500 | 0.4917 | 0.7443 | | 0.4668 | 1.49 | 17750 | 0.4976 | 0.7460 | | 0.4871 | 1.51 | 18000 | 0.4884 | 0.7446 | | 0.4843 | 1.53 | 18250 | 0.4884 | 0.7448 | | 0.4896 | 1.55 | 18500 | 0.4822 | 0.7461 | | 0.4483 | 1.57 | 18750 | 0.4855 | 0.7452 | | 0.5002 | 1.59 | 19000 | 0.4836 | 0.7469 | | 0.4795 | 1.61 | 19250 | 0.4819 | 0.7482 | | 0.4611 | 1.63 | 19500 | 0.4821 | 0.7475 | | 0.4657 | 1.65 | 19750 | 0.4832 | 0.7478 | | 0.492 | 1.67 | 20000 | 0.4808 | 0.7471 | | 0.495 | 1.7 | 20250 | 0.4813 | 0.7473 | | 0.467 | 1.72 | 20500 | 0.4838 | 0.7482 | | 0.4541 | 1.74 | 20750 | 0.4863 | 0.7482 | | 0.4823 | 1.76 | 21000 | 0.4887 | 0.7486 | | 0.4216 | 1.78 | 21250 | 0.4929 | 0.7469 | | 0.46 | 1.8 | 21500 | 0.4920 | 0.7469 | | 0.4548 | 1.82 | 21750 | 0.4927 | 0.7471 | | 0.4869 | 1.84 | 22000 | 0.4930 | 0.7473 | | 0.4919 | 1.86 | 22250 | 0.4927 | 0.7467 | | 0.4912 | 1.88 | 22500 | 0.4927 | 0.7461 | | 0.4843 | 1.9 | 22750 | 0.4927 | 0.7471 | | 0.5049 | 1.93 | 23000 | 0.4922 | 0.7458 | | 0.4681 | 1.95 | 23250 | 0.4925 | 0.7463 | | 0.4991 | 1.97 | 23500 | 0.4922 | 0.7467 | | 0.4893 | 1.99 | 23750 | 0.4920 | 0.7471 | ### Framework versions - PEFT 0.9.0 - Transformers 4.38.2 - Pytorch 2.1.2 - Datasets 2.18.0 - Tokenizers 0.15.2