GreenBitAI/DeepSeek-R1-Distill-Llama-70B-layer-mix-bpw-4.0-mlx
Updated
•
19
Haojin Yang
yanghaojin
AI & ML interests
None yet
Recent Activity
published
a model
4 days ago
GreenBitAI/DeepSeek-R1-Distill-Llama-70B-layer-mix-bpw-4.0-mlx
published
a model
10 days ago
GreenBitAI/DeepSeek-R1-Distill-Qwen-32B-layer-mix-bpw-4.0-mlx
upvoted
an
article
9 months ago
GPU Poor Savior: Revolutionizing Low-Bit Open Source LLMs and Cost-Effective Edge Computing
Organizations
yanghaojin's activity
published
a
model
4 days ago
published
a
model
10 days ago
upvoted
an
article
9 months ago
Article
GPU Poor Savior: Revolutionizing Low-Bit Open Source LLMs and Cost-Effective Edge Computing
By
•
•
10
upvoted
7
collections
9 months ago
GreenBitAI Llama-2
Collection
GreenBitAI's Llama 2 family in low-bit format
•
18 items
•
Updated
•
1
GreenBitAI Qwen1.5
Collection
GreenBitAI's Qwen1.5 family in low-bit format
•
74 items
•
Updated
•
2
reacted to
their
post with 🔥❤️🧠🚀
9 months ago
Post
900
Dear community,
Please check our recent blog post, "GPU Poor Savior: Revolutionizing Low-Bit Open Source LLMs and Cost-Effective Edge Computing". A cheaper and more efficient SFT scheme for quantized LLMs is provided.
https://huggingface.co./blog/NicoNico/green-bit-llm
Please check our recent blog post, "GPU Poor Savior: Revolutionizing Low-Bit Open Source LLMs and Cost-Effective Edge Computing". A cheaper and more efficient SFT scheme for quantized LLMs is provided.
https://huggingface.co./blog/NicoNico/green-bit-llm
posted
an
update
9 months ago
Post
900
Dear community,
Please check our recent blog post, "GPU Poor Savior: Revolutionizing Low-Bit Open Source LLMs and Cost-Effective Edge Computing". A cheaper and more efficient SFT scheme for quantized LLMs is provided.
https://huggingface.co./blog/NicoNico/green-bit-llm
Please check our recent blog post, "GPU Poor Savior: Revolutionizing Low-Bit Open Source LLMs and Cost-Effective Edge Computing". A cheaper and more efficient SFT scheme for quantized LLMs is provided.
https://huggingface.co./blog/NicoNico/green-bit-llm
reacted to
their
post with 🚀🔥
10 months ago
Post
2234
Full parameter fine-tuning of the LLaMA-3 8B model using a single GTX 3090 GPU with 24GB of graphics memory?
Please check out our tool for fine-tuning, inferencing, and evaluating GreenBitAI's low-bit LLMs:
https://github.com/GreenBitAI/green-bit-llm
Model Zoo:
https://huggingface.co./GreenBitAI
Please check out our tool for fine-tuning, inferencing, and evaluating GreenBitAI's low-bit LLMs:
https://github.com/GreenBitAI/green-bit-llm
Model Zoo:
https://huggingface.co./GreenBitAI
-
- 3 replies
replied to
their
post
10 months ago
Command for reproducing this run 😉 :
CUDA_VISIBLE_DEVICES=0 WANDB_DISABLED=true python -m sft.finetune --model GreenBitAI/Llama-3-8B-layer-mix-bpw-2.2 --tune-qweight-only --galore --galore-rank 64 --optimizer adamw8bit --batch-size 1 --seqlen 96
posted
an
update
10 months ago
Post
2234
Full parameter fine-tuning of the LLaMA-3 8B model using a single GTX 3090 GPU with 24GB of graphics memory?
Please check out our tool for fine-tuning, inferencing, and evaluating GreenBitAI's low-bit LLMs:
https://github.com/GreenBitAI/green-bit-llm
Model Zoo:
https://huggingface.co./GreenBitAI
Please check out our tool for fine-tuning, inferencing, and evaluating GreenBitAI's low-bit LLMs:
https://github.com/GreenBitAI/green-bit-llm
Model Zoo:
https://huggingface.co./GreenBitAI
-
-
- 3 replies
reacted to
their
post with 🔥
10 months ago
Post
1348
Dear all,
We are happy to share that we have just open-sourced over 200 low-bit LLMs. For the MLX community, we have prepared 2-4 bit versions of mainstream LLMs. You can visit the following collection to access them: GreenBitAI/greenbitai-mlx-llm-6614eb6ceb8da657c2b4ed58.
These low-bit models can be conveniently used through our open-source tool at https://github.com/GreenBitAI/gbx-lm.
Compared to other open-source quantization algorithms, these models provide better accuracy retention. We have provided some model evaluation results here:
https://github.com/GreenBitAI/green-bit-llm/blob/main/green_bit_llm/evaluation/README.md.
You can also evaluate the models yourself using the evaluation script we provided.
We are happy to share that we have just open-sourced over 200 low-bit LLMs. For the MLX community, we have prepared 2-4 bit versions of mainstream LLMs. You can visit the following collection to access them: GreenBitAI/greenbitai-mlx-llm-6614eb6ceb8da657c2b4ed58.
These low-bit models can be conveniently used through our open-source tool at https://github.com/GreenBitAI/gbx-lm.
Compared to other open-source quantization algorithms, these models provide better accuracy retention. We have provided some model evaluation results here:
https://github.com/GreenBitAI/green-bit-llm/blob/main/green_bit_llm/evaluation/README.md.
You can also evaluate the models yourself using the evaluation script we provided.
- 1 reply