Set "use_cache": true for a speedup.

by autobots - opened Apr 29, 2023

Apr 29, 2023

Not much of one but at least it makes it somewhat usable.

Jul 6, 2023

Same here!
It gave about 1 token/s.
However, Text genertion web UI gave me about 7 tokens/s.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment