Hugging Face's logo

minyichen
/
Llama-3-Taiwan-70B-Instruct-fp8

Text Generation
Transformers
Safetensors
Chinese
English
llama
llama-3
conversational
text-generation-inference
Inference Endpoints
fp8
Model card Files Community

Llama-3-Taiwan-70B-Instruct-fp8

Description

This repo contains fp8 model files for Llama-3-Taiwan-70B-Instruct.

Quantization parameter

  • activation_scheme : static
  • quant_method : fp8
  • ignored_layers : lm_head

It tooks about 8.5 hrs to quantize on H100.

Downloads last month
46
Safetensors
Model size
70.6B params
Tensor type
BF16
·
F8_E4M3
·
Inference Providers NEW
Text Generation
This model is not currently available via any of the supported Inference Providers.

Model tree for minyichen/Llama-3-Taiwan-70B-Instruct-fp8

Base model

meta-llama/Meta-Llama-3-70B
Finetuned
yentinglin/Llama-3-Taiwan-70B-Instruct
Quantized
(6)
this model

Collection including minyichen/Llama-3-Taiwan-70B-Instruct-fp8