googcheng/7b-lora

模型功能: 古诗句接龙 based on baichuan-inc/Baichuan-7B

Training procedure

CUDA_VISIBLE_DEVICES=0 python src/train_bash.py     --stage sft     --model_name_or_path baichuan-inc/Baichuan-7B     --do_train     --dataset alpaca_gpt4_zh     --template default     --finetuning_type lora     --output_dir path_to_sft_checkpoint     --overwrite_cache     --per_device_train_batch_size 4     --gradient_accumulation_steps 4     --lr_scheduler_type cosine     --logging_steps 10     --save_steps 1000     --learning_rate 5e-5     --num_train_epochs 20     --plot_loss     --fp16  --lora_target W_pack

Framework versions

• PEFT 0.4.0

Test