OPEA
/

Meta-Llama-3.1-405B-Instruct-int4-sym-inc

4-bit precision

intel/auto-round

Model card Files Files and versions Community

wenhuach commited on Dec 17, 2024

Commit

417447c

·

1 Parent(s): cb38f4a

change to fp16

Signed-off-by: wenhuach <[email protected]>

Files changed (1) hide show

config.json +1 -1

config.json CHANGED Viewed

@@ -51,7 +51,7 @@
   "rope_scaling": null,
   "rope_theta": 500000.0,
   "tie_word_embeddings": false,
-  "torch_dtype": "bfloat16",
   "transformers_version": "4.47.0",
   "use_cache": true,
   "vocab_size": 128256

   "rope_scaling": null,
   "rope_theta": 500000.0,
   "tie_word_embeddings": false,
+  "torch_dtype": "float16",
   "transformers_version": "4.47.0",
   "use_cache": true,
   "vocab_size": 128256