End of training

README.md

@@ -0,0 +1,88 @@
+---
+license: mit
+library_name: peft
+tags:
+- trl
+- sft
+- generated_from_trainer
+base_model: openai-community/gpt2
+model-index:
+- name: outputs
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# outputs
+
+This model is a fine-tuned version of [openai-community/gpt2](https://huggingface.co/openai-community/gpt2) on the None dataset.
+It achieves the following results on the evaluation set:
+- Loss: 1.0494
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 5e-05
+- train_batch_size: 1
+- eval_batch_size: 8
+- seed: 42
+- gradient_accumulation_steps: 4
+- total_train_batch_size: 4
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- lr_scheduler_warmup_steps: 0.03
+- num_epochs: 100
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss |
+|:-------------:|:-----:|:----:|:---------------:|
+| 2.9872        | 4.0   | 184  | 1.7517          |
+| 1.769         | 8.0   | 368  | 1.3586          |
+| 1.5071        | 12.0  | 552  | 1.2644          |
+| 1.4096        | 16.0  | 736  | 1.2088          |
+| 1.3405        | 20.0  | 920  | 1.1738          |
+| 1.2971        | 24.0  | 1104 | 1.1478          |
+| 1.2609        | 28.0  | 1288 | 1.1241          |
+| 1.2328        | 32.0  | 1472 | 1.1112          |
+| 1.2087        | 36.0  | 1656 | 1.0996          |
+| 1.1919        | 40.0  | 1840 | 1.0918          |
+| 1.1687        | 44.0  | 2024 | 1.0868          |
+| 1.157         | 48.0  | 2208 | 1.0768          |
+| 1.1436        | 52.0  | 2392 | 1.0724          |
+| 1.1296        | 56.0  | 2576 | 1.0684          |
+| 1.1147        | 60.0  | 2760 | 1.0657          |
+| 1.1082        | 64.0  | 2944 | 1.0622          |
+| 1.102         | 68.0  | 3128 | 1.0578          |
+| 1.0925        | 72.0  | 3312 | 1.0573          |
+| 1.0865        | 76.0  | 3496 | 1.0533          |
+| 1.0786        | 80.0  | 3680 | 1.0534          |
+| 1.076         | 84.0  | 3864 | 1.0523          |
+| 1.0764        | 88.0  | 4048 | 1.0505          |
+| 1.0685        | 92.0  | 4232 | 1.0495          |
+| 1.0684        | 96.0  | 4416 | 1.0489          |
+| 1.0662        | 100.0 | 4600 | 1.0494          |
+
+
+### Framework versions
+
+- PEFT 0.10.0
+- Transformers 4.39.3
+- Pytorch 2.2.1+cu121
+- Datasets 2.18.0
+- Tokenizers 0.15.2

adapter_config.json

@@ -0,0 +1,28 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "openai-community/gpt2",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "c_attn"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c206e633bab631548404dc94cd2ff176efc7deb16edd230e7b9b19e86833c9db
+size 1182680

runs/Apr09_12-52-42_0ce968f49445/events.out.tfevents.1712667163.0ce968f49445.186.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:926a5bc103bc34e48b017aab7a3dcc001ef04cb2ff83c4e67f0ea378c2f7c222
+size 5312

runs/Apr09_12-54-47_0ce968f49445/events.out.tfevents.1712667288.0ce968f49445.186.1

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fafa9c4e7793ccf063d61eb088f59ab392dd93bb068f3b41f1b33df059af43f5
+size 5660

runs/Apr09_12-56-30_0ce968f49445/events.out.tfevents.1712667391.0ce968f49445.186.2

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6b04486a7fdd4dc8e4ecd914bc5c534511913d671d813e515afcdf897085d9cb
+size 5660

runs/Apr09_12-59-37_0ce968f49445/events.out.tfevents.1712667578.0ce968f49445.186.3

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5c525620dc3be1323ee261d47e84be0715bdea83db7f304ded5f4ee979c4b43b
+size 6621

runs/Apr09_13-16-06_0ce968f49445/events.out.tfevents.1712668567.0ce968f49445.186.4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6bc235e3693a43a1be6298e72ea38248c90f853c3b9e19c7c41ebfe94af0e3dc
+size 6621

runs/Apr09_13-21-03_0ce968f49445/events.out.tfevents.1712668864.0ce968f49445.186.5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:75b7703bb318bfd667e2e59d358dcb35f68fef405a53f0fd092642763cd3d742
+size 6616

runs/Apr09_13-29-52_0ce968f49445/events.out.tfevents.1712669393.0ce968f49445.186.6

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:713d9a89a4141cc99c0cfd8910a62ecbef8a97859337f2b21872b2d53876b3bc
+size 115375

runs/Apr09_13-36-46_0ce968f49445/events.out.tfevents.1712669807.0ce968f49445.186.7

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5d0cb81e3b25e2f34d65509a8733f724a9390d567bbf1153b9417356de98e52d
+size 51826

runs/Apr09_13-49-15_0ce968f49445/events.out.tfevents.1712670556.0ce968f49445.186.8

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9ec84822eed16262c7ad6884ef197e79fb2e462991ea240b93b780edc9bc75c6
+size 6628

runs/Apr09_13-54-29_0ce968f49445/events.out.tfevents.1712670870.0ce968f49445.186.9

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ebbbb5ce6ee4b2e96b5850c6a154d083b7b5f83053540093e91f02d4b1edb544
+size 17720

special_tokens_map.json

@@ -0,0 +1,6 @@
+{
+  "bos_token": "<|endoftext|>",
+  "eos_token": "<|endoftext|>",
+  "pad_token": "<|endoftext|>",
+  "unk_token": "<|endoftext|>"
+}

tokenizer_config.json

@@ -0,0 +1,20 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "50256": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|endoftext|>",
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "<|endoftext|>",
+  "model_max_length": 1024,
+  "pad_token": "<|endoftext|>",
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|endoftext|>"
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bf5b87ed135761bd7e29daf2f2a0241915db0ab13b608fb1bc355aaf48392997
+size 4920