colab
#1
by
sdyy
- opened
it not run in colab t4
16 vram
12 ram
Greetings, thank you for your comment. I believe a 4-bit quantized version would require approximately 20 GB of VRAM to run the model efficiently. This is significantly less than the original model, which requires approximately 64-80 GB in full precision form.Similarly, for RAM, I would recommend a minimum of 16 GB. A NVIDIA RTX 3090 or RTX 4090 would suffice.
P.S: As of now, I have not tested it; I will run the tests and update this comment.
Satwik11
changed discussion status to
closed