Spaces:

HuggingFaceH4

/

falcon-chat

like 554

Running

Hello, I see performance boost and reduced VRAM Consumption with qlora implementation in recent transformers package (v4.30.1).

Activating bitsandbytes config which enabling nf4 quant type, load_in_4bit, and bfloat16.

Hopefully it will implement in this space. Thank you.