Spaces:

gustavoaq
/

llama_7b_ft_2

Paused

gustavoaq commited on Apr 8, 2023

Commit

2fa3303

1 Parent(s): 7a9e7c3

Update finetune.py

Files changed (1) hide show

finetune.py CHANGED Viewed

@@ -65,7 +65,8 @@ model = LlamaForCausalLM.from_pretrained(
 total_params, params = 0, 0
 tokenizer = LlamaTokenizer.from_pretrained(
-    "decapoda-research/llama-{}-hf".format(size), add_eos_token=True
 )
 model = prepare_model_for_int8_training(model)

 total_params, params = 0, 0
 tokenizer = LlamaTokenizer.from_pretrained(
+    "decapoda-research/llama-{}-hf".format(size), add_eos_token=True,
+    load_in_8bit_fp32_cpu_offload=True, device_map={0: [0]},
 )
 model = prepare_model_for_int8_training(model)