Abhaykoul commited on
Commit
fccb2f4
1 Parent(s): bc49d36

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +19 -4
README.md CHANGED
@@ -13,7 +13,7 @@ tags:
13
  #### Description
14
 
15
  Optimize your engagement with [This project](https://huggingface.co/OEvortex/HelpingAI-Lite) by seamlessly integrating GGUF Format model files.
16
-
17
  ### GGUF Technical Specifications
18
 
19
  Delve into the intricacies of GGUF, a meticulously crafted format that builds upon the robust foundation of the GGJT model. Tailored for heightened extensibility and user-centric functionality, GGUF introduces a suite of indispensable features:
@@ -30,8 +30,23 @@ Delve into the intricacies of GGUF, a meticulously crafted format that builds up
30
 
31
  The differentiator between GGJT and GGUF lies in the deliberate adoption of a key-value structure for hyperparameters (now termed metadata). Bid farewell to untyped lists, and embrace a structured approach that seamlessly accommodates new metadata without compromising compatibility with existing models. Augment your model with supplementary information for enhanced inference and model identification.
32
 
33
- ### Quantization
34
 
35
- Embark on a meticulous exploration of quantization with an array of well-defined methods at your disposal:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
36
 
37
- QUANTIZATION = q2_k, q3_k_l, q3_k_m, q3_k_s, q4_0, q4_1, q4_k_m, q4_k_s, q5_0, q5_1, q5_k_m, q5_k_s, q6_k, q8_0
 
13
  #### Description
14
 
15
  Optimize your engagement with [This project](https://huggingface.co/OEvortex/HelpingAI-Lite) by seamlessly integrating GGUF Format model files.
16
+ Please Subscribe to my youtube channel [OEvortex](https://youtube.com/@OEvortex)
17
  ### GGUF Technical Specifications
18
 
19
  Delve into the intricacies of GGUF, a meticulously crafted format that builds upon the robust foundation of the GGJT model. Tailored for heightened extensibility and user-centric functionality, GGUF introduces a suite of indispensable features:
 
30
 
31
  The differentiator between GGJT and GGUF lies in the deliberate adoption of a key-value structure for hyperparameters (now termed metadata). Bid farewell to untyped lists, and embrace a structured approach that seamlessly accommodates new metadata without compromising compatibility with existing models. Augment your model with supplementary information for enhanced inference and model identification.
32
 
 
33
 
34
+ **QUANTIZATION_METHODS:**
35
+
36
+ | Method | Quantization | Advantages | Trade-offs |
37
+ |---|---|---|---|
38
+ | q2_k | 2-bit integers | Significant model size reduction | Minimal impact on accuracy |
39
+ | q3_k_l | 3-bit integers | Balance between model size reduction and accuracy preservation | Moderate impact on accuracy |
40
+ | q3_k_m | 3-bit integers | Enhanced accuracy with mixed precision | Increased computational complexity |
41
+ | q3_k_s | 3-bit integers | Improved model efficiency with structured pruning | Reduced accuracy |
42
+ | q4_0 | 4-bit integers | Significant model size reduction | Moderate impact on accuracy |
43
+ | q4_1 | 4-bit integers | Enhanced accuracy with mixed precision | Increased computational complexity |
44
+ | q4_k_m | 4-bit integers | Optimized model size and accuracy with mixed precision and structured pruning | Reduced accuracy |
45
+ | q4_k_s | 4-bit integers | Improved model efficiency with structured pruning | Reduced accuracy |
46
+ | q5_0 | 5-bit integers | Balance between model size reduction and accuracy preservation | Moderate impact on accuracy |
47
+ | q5_1 | 5-bit integers | Enhanced accuracy with mixed precision | Increased computational complexity |
48
+ | q5_k_m | 5-bit integers | Optimized model size and accuracy with mixed precision and structured pruning | Reduced accuracy |
49
+ | q5_k_s | 5-bit integers | Improved model efficiency with structured pruning | Reduced accuracy |
50
+ | q6_k | 6-bit integers | Balance between model size reduction and accuracy preservation | Moderate impact on accuracy |
51
+ | q8_0 | 8-bit integers | Significant model size reduction | Minimal impact on accuracy |
52