Update README.md
Browse files
README.md
CHANGED
@@ -267,7 +267,9 @@ dataset.select_columns(["conversations"]).to_json("/workspace/airoboros-3.2_plus
|
|
267 |
## Training
|
268 |
The Jamba-v0.1 base model was trained for roughly 3 hours in a A100 (80GB) x 4 environment on the Azure cloud (Standard_NC96ads_A100_v4).
|
269 |
|
270 |
-
|
|
|
|
|
271 |
|
272 |
<details>
|
273 |
<summary>Training config</summary>
|
|
|
267 |
## Training
|
268 |
The Jamba-v0.1 base model was trained for roughly 3 hours in a A100 (80GB) x 4 environment on the Azure cloud (Standard_NC96ads_A100_v4).
|
269 |
|
270 |
+
We trained using QLoRA and merged the adapter to the original weights.
|
271 |
+
|
272 |
+
Our training harness was Axolotl using the ChatML chat template. Full details of the training config are below:
|
273 |
|
274 |
<details>
|
275 |
<summary>Training config</summary>
|