hughtayloe
/

handertrails

Inference Endpoints

Model card Files Files and versions Community

hughtayloe commited on Feb 7

Commit

1a792e3

•

1 Parent(s): 185c874

Update handler.py

Files changed (1) hide show

handler.py +1 -1

handler.py CHANGED Viewed

@@ -12,7 +12,7 @@ class EndpointHandler():
         self.model = LlavaForConditionalGeneration.from_pretrained(
             model_id,
             torch_dtype=torch.float16,
-            low_cpu_mem_usage=True,
             load_in_4bit=True
         )
         self.processor = AutoProcessor.from_pretrained(model_id)

         self.model = LlavaForConditionalGeneration.from_pretrained(
             model_id,
             torch_dtype=torch.float16,
+            low_cpu_mem_usage=True,
             load_in_4bit=True
         )
         self.processor = AutoProcessor.from_pretrained(model_id)