Hugging Face's logo

tttx
/
problem12_model_more_aug_30

PEFT
Safetensors
llama
alignment-handbook
trl
sft
Generated from Trainer
Model card Files Community
aadityap commited on
Commit
89bb8aa
•
1 Parent(s): 60879bd

Model save

Browse files
Files changed (4) hide show
  1. README.md +64 -0
  2. all_results.json +9 -0
  3. train_results.json +9 -0
  4. trainer_state.json +86 -0
README.md ADDED
@@ -0,0 +1,64 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: barc0/Llama-3.1-ARC-Potpourri-Transduction-8B
3
+ library_name: peft
4
+ license: llama3.1
5
+ tags:
6
+ - trl
7
+ - sft
8
+ - generated_from_trainer
9
+ model-index:
10
+ - name: problem12_model_more_aug_30
11
+ results: []
12
+ ---
13
+
14
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
15
+ should probably proofread and complete it, then remove this comment. -->
16
+
17
+ # problem12_model_more_aug_30
18
+
19
+ This model is a fine-tuned version of [barc0/Llama-3.1-ARC-Potpourri-Transduction-8B](https://huggingface.co/barc0/Llama-3.1-ARC-Potpourri-Transduction-8B) on an unknown dataset.
20
+ It achieves the following results on the evaluation set:
21
+ - Loss: nan
22
+
23
+ ## Model description
24
+
25
+ More information needed
26
+
27
+ ## Intended uses & limitations
28
+
29
+ More information needed
30
+
31
+ ## Training and evaluation data
32
+
33
+ More information needed
34
+
35
+ ## Training procedure
36
+
37
+ ### Training hyperparameters
38
+
39
+ The following hyperparameters were used during training:
40
+ - learning_rate: 5e-05
41
+ - train_batch_size: 2
42
+ - eval_batch_size: 2
43
+ - seed: 42
44
+ - distributed_type: multi-GPU
45
+ - optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
46
+ - lr_scheduler_type: cosine
47
+ - lr_scheduler_warmup_ratio: 0.1
48
+ - num_epochs: 2
49
+
50
+ ### Training results
51
+
52
+ | Training Loss | Epoch | Step | Validation Loss |
53
+ |:-------------:|:-----:|:----:|:---------------:|
54
+ | 0.0 | 1.0 | 2 | nan |
55
+ | 0.002 | 2.0 | 4 | nan |
56
+
57
+
58
+ ### Framework versions
59
+
60
+ - PEFT 0.13.2
61
+ - Transformers 4.47.0.dev0
62
+ - Pytorch 2.4.0+cu121
63
+ - Datasets 3.1.0
64
+ - Tokenizers 0.20.3
all_results.json ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 2.0,
3
+ "total_flos": 97712209920.0,
4
+ "train_loss": 0.0005083280047983862,
5
+ "train_runtime": 55.9946,
6
+ "train_samples": 4,
7
+ "train_samples_per_second": 0.143,
8
+ "train_steps_per_second": 0.071
9
+ }
train_results.json ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 2.0,
3
+ "total_flos": 97712209920.0,
4
+ "train_loss": 0.0005083280047983862,
5
+ "train_runtime": 55.9946,
6
+ "train_samples": 4,
7
+ "train_samples_per_second": 0.143,
8
+ "train_steps_per_second": 0.071
9
+ }
trainer_state.json ADDED
@@ -0,0 +1,86 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_metric": null,
3
+ "best_model_checkpoint": null,
4
+ "epoch": 2.0,
5
+ "eval_steps": 500,
6
+ "global_step": 4,
7
+ "is_hyper_param_search": false,
8
+ "is_local_process_zero": true,
9
+ "is_world_process_zero": true,
10
+ "log_history": [
11
+ {
12
+ "epoch": 0.5,
13
+ "grad_norm": 0.021461592006973486,
14
+ "learning_rate": 5e-05,
15
+ "loss": 0.0029,
16
+ "step": 1
17
+ },
18
+ {
19
+ "epoch": 1.0,
20
+ "grad_norm": 0.0002054731229294204,
21
+ "learning_rate": 3.7500000000000003e-05,
22
+ "loss": 0.0,
23
+ "step": 2
24
+ },
25
+ {
26
+ "epoch": 1.0,
27
+ "eval_loss": NaN,
28
+ "eval_runtime": 0.9406,
29
+ "eval_samples_per_second": 1.063,
30
+ "eval_steps_per_second": 1.063,
31
+ "step": 2
32
+ },
33
+ {
34
+ "epoch": 1.5,
35
+ "grad_norm": 0.00021730421010002378,
36
+ "learning_rate": 1.2500000000000006e-05,
37
+ "loss": 0.0,
38
+ "step": 3
39
+ },
40
+ {
41
+ "epoch": 2.0,
42
+ "grad_norm": 0.015960496683072516,
43
+ "learning_rate": 0.0,
44
+ "loss": 0.002,
45
+ "step": 4
46
+ },
47
+ {
48
+ "epoch": 2.0,
49
+ "eval_loss": NaN,
50
+ "eval_runtime": 0.9501,
51
+ "eval_samples_per_second": 1.053,
52
+ "eval_steps_per_second": 1.053,
53
+ "step": 4
54
+ },
55
+ {
56
+ "epoch": 2.0,
57
+ "step": 4,
58
+ "total_flos": 97712209920.0,
59
+ "train_loss": 0.0005083280047983862,
60
+ "train_runtime": 55.9946,
61
+ "train_samples_per_second": 0.143,
62
+ "train_steps_per_second": 0.071
63
+ }
64
+ ],
65
+ "logging_steps": 1,
66
+ "max_steps": 4,
67
+ "num_input_tokens_seen": 0,
68
+ "num_train_epochs": 2,
69
+ "save_steps": 500,
70
+ "stateful_callbacks": {
71
+ "TrainerControl": {
72
+ "args": {
73
+ "should_epoch_stop": false,
74
+ "should_evaluate": false,
75
+ "should_log": false,
76
+ "should_save": true,
77
+ "should_training_stop": true
78
+ },
79
+ "attributes": {}
80
+ }
81
+ },
82
+ "total_flos": 97712209920.0,
83
+ "train_batch_size": 2,
84
+ "trial_name": null,
85
+ "trial_params": null
86
+ }