Qwen2.5-VL-7B-Instruct_arc-agi-transduction100k-direct-ft

This model is a fine-tuned version of Qwen/Qwen2.5-VL-7B-Instruct on the arc-agi-transduction100k-direct-ft dataset. It achieves the following results on the evaluation set:

• Loss: 0.0003

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 2
• eval_batch_size: 1
• seed: 42
• distributed_type: multi-GPU
• num_devices: 8
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• total_eval_batch_size: 8
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 3.0

Training results

Training Loss	Epoch	Step	Validation Loss
0.0177	0.3549	1000	0.0153
0.0086	0.7098	2000	0.0066
0.0055	1.0646	3000	0.0048
0.0039	1.4195	4000	0.0026
0.0018	1.7744	5000	0.0019
0.0007	2.1292	6000	0.0008
0.0004	2.4841	7000	0.0004
0.0002	2.8390	8000	0.0003

Framework versions

• Transformers 4.50.0.dev0
• Pytorch 2.5.1+cu124
• Datasets 3.0.2
• Tokenizers 0.21.0