Text Generation
Transformers
Safetensors
jamba
conversational
custom_code
Inference Endpoints
ptrdvn commited on
Commit
6cb9135
1 Parent(s): a51f4b9

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -1
README.md CHANGED
@@ -267,7 +267,9 @@ dataset.select_columns(["conversations"]).to_json("/workspace/airoboros-3.2_plus
267
  ## Training
268
  The Jamba-v0.1 base model was trained for roughly 3 hours in a A100 (80GB) x 4 environment on the Azure cloud (Standard_NC96ads_A100_v4).
269
 
270
- Our training harness was Axolotl, with the following config as our training parameters:
 
 
271
 
272
  <details>
273
  <summary>Training config</summary>
 
267
  ## Training
268
  The Jamba-v0.1 base model was trained for roughly 3 hours in a A100 (80GB) x 4 environment on the Azure cloud (Standard_NC96ads_A100_v4).
269
 
270
+ We trained using QLoRA and merged the adapter to the original weights.
271
+
272
+ Our training harness was Axolotl using the ChatML chat template. Full details of the training config are below:
273
 
274
  <details>
275
  <summary>Training config</summary>