hughtayloe
/

handertrails

Inference Endpoints

Model card Files Files and versions Community

hughtayloe commited on Feb 7

Commit

185c874

•

1 Parent(s): c16b164

Update handler.py

Files changed (1) hide show

handler.py +1 -1

handler.py CHANGED Viewed

@@ -14,7 +14,7 @@ class EndpointHandler():
             torch_dtype=torch.float16,
             low_cpu_mem_usage=True,
             load_in_4bit=True
-        ).to(0)
         self.processor = AutoProcessor.from_pretrained(model_id)
     def __call__(self, data: Dict[str, Any]):

             torch_dtype=torch.float16,
             low_cpu_mem_usage=True,
             load_in_4bit=True
+        )
         self.processor = AutoProcessor.from_pretrained(model_id)
     def __call__(self, data: Dict[str, Any]):