Update README.md
Browse filesTest for permission
README.md
CHANGED
@@ -34,7 +34,7 @@ prompt_template: '<|system|>
|
|
34 |
|
35 |
This repository contains [AWS Inferentia2](https://aws.amazon.com/ec2/instance-types/inf2/) and [neuronx](https://awsdocs-neuron.readthedocs-hosted.com/en/latest/) compatible checkpoints for [HuggingFaceH4/zephyr-7b-beta](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta).
|
36 |
|
37 |
-
However, this file includes an example of how to compile various versions of Zephyr. Support isn’t available yet (as of 1/
|
38 |
|
39 |
These instructions closely follow the [Developer Guide](https://awsdocs-neuron.readthedocs-hosted.com/en/latest/libraries/transformers-neuronx/transformers-neuronx-developer-guide.html#grouped-query-attention-gqa-support-beta). Look there for more detailed explanations, especially for the GQA settings.
|
40 |
|
|
|
34 |
|
35 |
This repository contains [AWS Inferentia2](https://aws.amazon.com/ec2/instance-types/inf2/) and [neuronx](https://awsdocs-neuron.readthedocs-hosted.com/en/latest/) compatible checkpoints for [HuggingFaceH4/zephyr-7b-beta](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta).
|
36 |
|
37 |
+
However, this file includes an example of how to compile various versions of Zephyr. Support isn’t available yet (as of 1/10/2024) in the [optimum neuron](https://huggingface.co/docs/optimum-neuron/index) framework, so we use the base transformers library.
|
38 |
|
39 |
These instructions closely follow the [Developer Guide](https://awsdocs-neuron.readthedocs-hosted.com/en/latest/libraries/transformers-neuronx/transformers-neuronx-developer-guide.html#grouped-query-attention-gqa-support-beta). Look there for more detailed explanations, especially for the GQA settings.
|
40 |
|