Hugging Face's logo

Spaces:
olivierdehaene
/
chat-llm-streaming
Runtime error

App Files Community
15

Can we add https://huggingface.co./BlinkDL/rwkv-4-pile-14b/blob/main/RWKV-4-Pile-14B-20230313-ctx8192-test1050.pth

#6
by BlinkDL - opened
Discussion
BlinkDL

You can use https://pypi.org/project/rwkv/ :)

It's already a strong chatbot without finetuning and supports INT8 too: https://github.com/BlinkDL/ChatRWKV

Chat prompts: https://github.com/BlinkDL/ChatRWKV/tree/main/v2/prompt/default

olivierdehaene
 Owner

Yes that's something I want to work on.

  • What is the status of the PR integrating it to transformers?
  • What is the VRAM budget of the largest version of the model?

The best would be to open an issue here: https://github.com/huggingface/text-generation-inference/issues

BlinkDL

HF Integration: Takes time as the community is working on it (slowly).

VRAM budget: 16G is enough for 14B ctx8192 INT8 :)

I am training a new version on Pile v2 (my version: 1.73T tokens, 30% code).

BlinkDL changed discussion status to closed
BlinkDL changed discussion status to open

Sign up or log in to comment