munish0838 commited on
Commit
8a35aa9
1 Parent(s): 11ed952

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +73 -0
README.md ADDED
@@ -0,0 +1,73 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ pipeline_tag: text-generation
3
+ tags:
4
+ - llama
5
+ base_model: nota-ai/st-vicuna-v1.3-10.5b-ppl
6
+ ---
7
+
8
+ # QuantFactory/st-vicuna-v1.3-10.5b-ppl-GGUF
9
+ This is quantized version of [nota-ai/st-vicuna-v1.3-10.5b-ppl](https://huggingface.co/nota-ai/st-vicuna-v1.3-10.5b-ppl) created using llama.cpp
10
+
11
+ # Model Description
12
+ ### Shortened LLaMA Model Card
13
+
14
+ Shortened LLaMA is a depth-pruned version of LLaMA models & variants for efficient text generation.
15
+
16
+ - **Developed by:** [Nota AI](https://www.nota.ai/)
17
+ - **License:** Non-commercial license
18
+ - **Repository:** https://github.com/Nota-NetsPresso/shortened-llm
19
+ - **Paper:** https://arxiv.org/abs/2402.02834
20
+
21
+ ## Compression Method
22
+ After identifying unimportant Transformer blocks, we perform one-shot pruning and light LoRA-based retraining.
23
+ <details>
24
+ <summary>
25
+ Click to see a method figure.
26
+ </summary>
27
+
28
+ <img alt="method" img src="https://netspresso-research-code-release.s3.us-east-2.amazonaws.com/compressed-llm/st-llama_method.png" width="100%">
29
+
30
+ </details>
31
+
32
+ ## Model Links
33
+ | Source<br>Model | Pruning<br>Ratio | Pruning<br>Criterion | HF Models<br>Link |
34
+ |:---:|:---:|:---:|:---:|
35
+ | LLaMA-1-7B | 20% | PPL | [nota-ai/st-llama-1-5.5b-ppl](https://huggingface.co/nota-ai/st-llama-1-5.5b-ppl) |
36
+ | LLaMA-1-7B | 20% | Taylor+ | [nota-ai/st-llama-1-5.5b-taylor](https://huggingface.co/nota-ai/st-llama-1-5.5b-taylor) |
37
+ | Vicuna-v1.3-7B | 20% | PPL | [nota-ai/st-vicuna-v1.3-5.5b-ppl](https://huggingface.co/nota-ai/st-vicuna-v1.3-5.5b-ppl) |
38
+ | Vicuna-v1.3-7B | 20% | Taylor+ | [nota-ai/st-vicuna-v1.3-5.5b-taylor](https://huggingface.co/nota-ai/st-vicuna-v1.3-5.5b-taylor) |
39
+ | Vicuna-v1.3-13B | 21% | PPL | [nota-ai/st-vicuna-v1.3-10.5b-ppl](https://huggingface.co/nota-ai/st-vicuna-v1.3-10.5b-ppl) |
40
+ | Vicuna-v1.3-13B | 21% | Taylor+ | [nota-ai/st-vicuna-v1.3-10.5b-taylor](https://huggingface.co/nota-ai/st-vicuna-v1.3-10.5b-taylor) |
41
+
42
+ ## Zero-shot Performance & Efficiency Results
43
+ - EleutherAI/lm-evaluation-harness version [3326c54](https://github.com/EleutherAI/lm-evaluation-harness/tree/3326c547a733d598b4377e54be96e194861b964c)
44
+
45
+ <img alt="results" img src="https://netspresso-research-code-release.s3.us-east-2.amazonaws.com/compressed-llm/st-llama_zero-shot_scores.png" width="100%">
46
+
47
+ ## License
48
+ - All rights related to this repository and the compressed models are reserved by Nota Inc.
49
+ - The intended use is strictly limited to research and non-commercial projects.
50
+
51
+ ## Acknowledgments
52
+ - [LLM-Pruner](https://github.com/horseee/LLM-Pruner), which utilizes [LM Evaluation Harness](https://github.com/EleutherAI/lm-evaluation-harness), [PEFT](https://github.com/huggingface/peft), and [Alpaca-LoRA](https://github.com/tloen/alpaca-lora). Thanks for the pioneering work on structured pruning of LLMs!
53
+ - Meta AI's [LLaMA](https://github.com/facebookresearch/llama) and LMSYS Org's [Vicuna](https://github.com/lm-sys/FastChat/blob/main/docs/vicuna_weights_version.md). Thanks for the open-source LLMs!
54
+
55
+ ## Original Model Citation
56
+ ```bibtex
57
+ @article{kim2024shortened,
58
+ title={Shortened LLaMA: A Simple Depth Pruning for Large Language Models},
59
+ author={Kim, Bo-Kyeong and Kim, Geonmin and Kim, Tae-Ho and Castells, Thibault and Choi, Shinkook and Shin, Junho and Song, Hyoung-Kyu},
60
+ journal={arXiv preprint arXiv:2402.02834},
61
+ year={2024},
62
+ url={https://arxiv.org/abs/2402.02834}
63
+ }
64
+ ```
65
+ ```bibtex
66
+ @article{kim2024mefomo,
67
+ title={Shortened LLaMA: A Simple Depth Pruning for Large Language Models},
68
+ author={Kim, Bo-Kyeong and Kim, Geonmin and Kim, Tae-Ho and Castells, Thibault and Choi, Shinkook and Shin, Junho and Song, Hyoung-Kyu},
69
+ journal={ICLR Workshop on Mathematical and Empirical Understanding of Foundation Models (ME-FoMo)},
70
+ year={2024},
71
+ url={https://openreview.net/forum?id=18VGxuOdpu}
72
+ }
73
+ ```