--- license: mit library_name: peft tags: - trl - reward-trainer - generated_from_trainer metrics: - accuracy base_model: openai-community/gpt2-large model-index: - name: RM-HH-GPT2Large_helpful_gpt3_loraR64_40000_gpt2-large_shuffleTrue_extractchosenFalse results: [] --- # RM-HH-GPT2Large_helpful_gpt3_loraR64_40000_gpt2-large_shuffleTrue_extractchosenFalse This model is a fine-tuned version of [openai-community/gpt2-large](https://huggingface.co./openai-community/gpt2-large) on the None dataset. It achieves the following results on the evaluation set: - Loss: 0.4752 - Accuracy: 0.7446 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 1.41e-05 - train_batch_size: 1 - eval_batch_size: 8 - seed: 42 - gradient_accumulation_steps: 4 - total_train_batch_size: 4 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - num_epochs: 2.0 ### Training results | Training Loss | Epoch | Step | Validation Loss | Accuracy | |:-------------:|:-----:|:-----:|:---------------:|:--------:| | 0.6556 | 0.02 | 250 | 0.6162 | 0.6465 | | 0.5807 | 0.04 | 500 | 0.5552 | 0.6974 | | 0.5449 | 0.06 | 750 | 0.5322 | 0.7111 | | 0.5024 | 0.08 | 1000 | 0.5223 | 0.7145 | | 0.5227 | 0.1 | 1250 | 0.5184 | 0.7147 | | 0.52 | 0.13 | 1500 | 0.5156 | 0.7175 | | 0.5127 | 0.15 | 1750 | 0.5127 | 0.7143 | | 0.4988 | 0.17 | 2000 | 0.5081 | 0.7198 | | 0.5129 | 0.19 | 2250 | 0.5051 | 0.7217 | | 0.4995 | 0.21 | 2500 | 0.5028 | 0.7254 | | 0.508 | 0.23 | 2750 | 0.5009 | 0.7264 | | 0.4953 | 0.25 | 3000 | 0.4992 | 0.7277 | | 0.5049 | 0.27 | 3250 | 0.4986 | 0.7275 | | 0.5042 | 0.29 | 3500 | 0.4974 | 0.7309 | | 0.4971 | 0.31 | 3750 | 0.4958 | 0.7286 | | 0.4871 | 0.33 | 4000 | 0.4950 | 0.7316 | | 0.5001 | 0.36 | 4250 | 0.4938 | 0.7277 | | 0.507 | 0.38 | 4500 | 0.4927 | 0.7290 | | 0.5242 | 0.4 | 4750 | 0.4889 | 0.7320 | | 0.5212 | 0.42 | 5000 | 0.4876 | 0.7350 | | 0.4955 | 0.44 | 5250 | 0.4864 | 0.7354 | | 0.4809 | 0.46 | 5500 | 0.4884 | 0.7363 | | 0.4784 | 0.48 | 5750 | 0.4865 | 0.7326 | | 0.4933 | 0.5 | 6000 | 0.4857 | 0.7348 | | 0.5008 | 0.52 | 6250 | 0.4827 | 0.7335 | | 0.5045 | 0.54 | 6500 | 0.4816 | 0.7358 | | 0.4757 | 0.57 | 6750 | 0.4843 | 0.7356 | | 0.5055 | 0.59 | 7000 | 0.4819 | 0.7318 | | 0.4984 | 0.61 | 7250 | 0.4800 | 0.7362 | | 0.5073 | 0.63 | 7500 | 0.4813 | 0.7354 | | 0.4512 | 0.65 | 7750 | 0.4836 | 0.7375 | | 0.4841 | 0.67 | 8000 | 0.4830 | 0.7380 | | 0.4621 | 0.69 | 8250 | 0.4854 | 0.7392 | | 0.487 | 0.71 | 8500 | 0.4807 | 0.7388 | | 0.474 | 0.73 | 8750 | 0.4814 | 0.7422 | | 0.4921 | 0.75 | 9000 | 0.4826 | 0.7412 | | 0.4687 | 0.77 | 9250 | 0.4794 | 0.7401 | | 0.4793 | 0.8 | 9500 | 0.4780 | 0.7407 | | 0.4659 | 0.82 | 9750 | 0.4779 | 0.7399 | | 0.4821 | 0.84 | 10000 | 0.4774 | 0.7403 | | 0.499 | 0.86 | 10250 | 0.4786 | 0.7416 | | 0.5152 | 0.88 | 10500 | 0.4747 | 0.7394 | | 0.4566 | 0.9 | 10750 | 0.4764 | 0.7405 | | 0.45 | 0.92 | 11000 | 0.4761 | 0.7394 | | 0.4726 | 0.94 | 11250 | 0.4794 | 0.7397 | | 0.468 | 0.96 | 11500 | 0.4784 | 0.7390 | | 0.4951 | 0.98 | 11750 | 0.4744 | 0.7395 | | 0.5151 | 1.0 | 12000 | 0.4741 | 0.7382 | | 0.4632 | 1.03 | 12250 | 0.4751 | 0.7409 | | 0.4746 | 1.05 | 12500 | 0.4742 | 0.7422 | | 0.4858 | 1.07 | 12750 | 0.4743 | 0.7424 | | 0.482 | 1.09 | 13000 | 0.4751 | 0.7426 | | 0.4527 | 1.11 | 13250 | 0.4798 | 0.7431 | | 0.4709 | 1.13 | 13500 | 0.4773 | 0.7437 | | 0.457 | 1.15 | 13750 | 0.4779 | 0.7426 | | 0.4771 | 1.17 | 14000 | 0.4817 | 0.7414 | | 0.4755 | 1.19 | 14250 | 0.4792 | 0.7439 | | 0.4548 | 1.21 | 14500 | 0.4816 | 0.7446 | | 0.4459 | 1.23 | 14750 | 0.4841 | 0.7424 | | 0.4699 | 1.26 | 15000 | 0.4798 | 0.7409 | | 0.4373 | 1.28 | 15250 | 0.4848 | 0.7392 | | 0.4754 | 1.3 | 15500 | 0.4826 | 0.7409 | | 0.4806 | 1.32 | 15750 | 0.4792 | 0.7414 | | 0.4934 | 1.34 | 16000 | 0.4779 | 0.7431 | | 0.4501 | 1.36 | 16250 | 0.4797 | 0.7426 | | 0.4654 | 1.38 | 16500 | 0.4769 | 0.7439 | | 0.4839 | 1.4 | 16750 | 0.4763 | 0.7446 | | 0.4942 | 1.42 | 17000 | 0.4769 | 0.7429 | | 0.4391 | 1.44 | 17250 | 0.4810 | 0.7426 | | 0.4947 | 1.46 | 17500 | 0.4787 | 0.7422 | | 0.4644 | 1.49 | 17750 | 0.4819 | 0.7422 | | 0.4945 | 1.51 | 18000 | 0.4775 | 0.7420 | | 0.4333 | 1.53 | 18250 | 0.4830 | 0.7418 | | 0.4863 | 1.55 | 18500 | 0.4795 | 0.7416 | | 0.4843 | 1.57 | 18750 | 0.4777 | 0.7414 | | 0.4772 | 1.59 | 19000 | 0.4771 | 0.7427 | | 0.4834 | 1.61 | 19250 | 0.4765 | 0.7422 | | 0.4673 | 1.63 | 19500 | 0.4758 | 0.7424 | | 0.4795 | 1.65 | 19750 | 0.4763 | 0.7424 | | 0.4808 | 1.67 | 20000 | 0.4764 | 0.7427 | | 0.4593 | 1.7 | 20250 | 0.4772 | 0.7427 | | 0.4947 | 1.72 | 20500 | 0.4771 | 0.7431 | | 0.4776 | 1.74 | 20750 | 0.4770 | 0.7444 | | 0.4841 | 1.76 | 21000 | 0.4764 | 0.7452 | | 0.4536 | 1.78 | 21250 | 0.4766 | 0.7441 | | 0.4772 | 1.8 | 21500 | 0.4766 | 0.7444 | | 0.4668 | 1.82 | 21750 | 0.4766 | 0.7439 | | 0.4884 | 1.84 | 22000 | 0.4750 | 0.7444 | | 0.498 | 1.86 | 22250 | 0.4745 | 0.7446 | | 0.47 | 1.88 | 22500 | 0.4743 | 0.7443 | | 0.4524 | 1.9 | 22750 | 0.4752 | 0.7448 | | 0.4885 | 1.93 | 23000 | 0.4754 | 0.7441 | | 0.4734 | 1.95 | 23250 | 0.4752 | 0.7448 | | 0.4882 | 1.97 | 23500 | 0.4753 | 0.7446 | | 0.4929 | 1.99 | 23750 | 0.4752 | 0.7446 | ### Framework versions - PEFT 0.9.0 - Transformers 4.38.2 - Pytorch 2.1.2 - Datasets 2.18.0 - Tokenizers 0.15.2