Spaces:
Running
on
Zero
Running
on
Zero
Slow
#3
by
phaser10
- opened
Hello. The generation speed is so low that it must be running on a CPU, yet it drains ZeroGPU quota.
Any ideas on why the generation is so slow?
Thanks for reporting!
Well, it's slow but actually using GPU. You can check it by running this Space locally and monitoring GPU usage. My colleague who tried the transformers implementation also reported that it was slow, and
@ArthurZ
suggested that using torch.compile would make it faster, but torch.compile is not used in this Space because it's not compatible with ZeroGPU.
@ArthurZ
Do you have any other ideas why this Space is slow?