Spaces:

AdrienB134
/

rag_colpali_idefics3

Running on Zero

AdrienB134 commited on Sep 5, 2024

Commit

09ba206

1 Parent(s): 1784c2c

flas

Files changed (1) hide show

app.py CHANGED Viewed

@@ -36,7 +36,7 @@ def model_inference(
     id_model = Idefics3ForConditionalGeneration.from_pretrained("HuggingFaceM4/Idefics3-8B-Llama3",
             torch_dtype=torch.bfloat16,
-            #_attn_implementation="flash_attention_2"
                                                             ).to("cuda")
     BAD_WORDS_IDS = id_processor.tokenizer(["<image>", "<fake_token_around_image>"], add_special_tokens=False).input_ids

     id_model = Idefics3ForConditionalGeneration.from_pretrained("HuggingFaceM4/Idefics3-8B-Llama3",
             torch_dtype=torch.bfloat16,
+            attn_implementation="flash_attention_2"
                                                             ).to("cuda")
     BAD_WORDS_IDS = id_processor.tokenizer(["<image>", "<fake_token_around_image>"], add_special_tokens=False).input_ids