Update README.md
Browse files
README.md
CHANGED
@@ -9,7 +9,7 @@ base_model:
|
|
9 |
The Snowflake AI Research team is releasing a series of SwiftKV optimized Llama-3.1 models. [SwiftKV](https://arxiv.org/abs/2410.03960) is a series of inference optimizations that goes beyond traditional key-value (KV) cache compression. This method reduces computational overhead during prompt processing by combining model rewiring and knowledge-preserving self-distillation, allowing prefill tokens to skip up to half the model's layers. SwiftKV achieves up to 2x improvements in throughput, latency, and cost efficiency with minimal accuracy loss, making LLM deployments more performant and economically viable.
|
10 |
|
11 |
For more details about the technique
|
12 |
-
* Blog:
|
13 |
* arXiv paper: https://arxiv.org/abs/2410.03960
|
14 |
|
15 |
## Eval metrics
|
|
|
9 |
The Snowflake AI Research team is releasing a series of SwiftKV optimized Llama-3.1 models. [SwiftKV](https://arxiv.org/abs/2410.03960) is a series of inference optimizations that goes beyond traditional key-value (KV) cache compression. This method reduces computational overhead during prompt processing by combining model rewiring and knowledge-preserving self-distillation, allowing prefill tokens to skip up to half the model's layers. SwiftKV achieves up to 2x improvements in throughput, latency, and cost efficiency with minimal accuracy loss, making LLM deployments more performant and economically viable.
|
10 |
|
11 |
For more details about the technique
|
12 |
+
* Blog: https://www.snowflake.com/engineering-blog/swiftkv-llm-compute-reduction/
|
13 |
* arXiv paper: https://arxiv.org/abs/2410.03960
|
14 |
|
15 |
## Eval metrics
|