Hugging Face's logo

tokyotech-llm
/
Llama-3.1-Swallow-70B-Instruct-v0.1

Text Generation
Transformers
Safetensors
English
Japanese
llama
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
Taishi-N324 commited on
Commit
161b3d5
•
1 Parent(s): a7146d0

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -5
README.md CHANGED
@@ -68,14 +68,14 @@ The website [https://swallow-llm.github.io/](https://swallow-llm.github.io/) pro
68
  ||4-shot|4-shot|4-shot|4-shot|4-shot|5-shot|4-shot|3-shot|0-shot||
69
  ||Acc|EMacc|Acc|EMacc|Acc|Acc|EMacc|CoTEMAcc|pass@1||
70
  | Qwen2-72B-Instruct | 0.4360 | 0.7588 | 0.6857 | 0.3913 | 0.9110 | 0.8391 | 0.8499 | 0.2436 | 0.6939 | 0.6455 |
71
- | Qwen2.5-72B-Instruct | 0.4540 | 0.6764 | 0.7064 | 0.3550 | 0.8895 | 0.8478 | 0.9113 | 0.4027 | 0.6165 | 0.6511 |
72
  | Llama 3 70B Instruct | 0.4400 | 0.7999 | 0.6552 | 0.4024 | 0.9127 | 0.7992 | 0.9052 | 0.8326 | 0.7555 | 0.7225 |
73
- | Llama 3.1 70B Instruct | 0.4300 | 0.8212 | 0.6621 | 0.3921 | 0.9157 | 0.8213 | 0.8764 | 0.8390 | 0.7915 | 0.7277 |
74
  | Llama 3 Youko 70B Instruct | 0.4500 | 0.7973 | 0.6863 | 0.3914 | 0.9153 | 0.8055 | 0.8923 | 0.7814 | 0.6598 | 0.7088 |
75
  | Llama-3.1-70B-Japanese-Instruct-2407 | 0.4220 | 0.8104 | 0.6481 | 0.3744 | 0.9170 | 0.8071 | 0.8893 | 0.8228 | 0.7463 | 0.7153 |
76
- | Llama 3 heron brain 70B v0.3 | 0.4460 | 0.8107 | 0.6682 | 0.4085 | 0.9174 | 0.7898 | 0.8772 | 0.7586 | 0.6713 | 0.7053 |
77
- | Llama 3 Swallow 70B Instruct | 0.4520 | 0.8174 | 0.6758 | 0.4050 | 0.9230 | 0.7883 | 0.8688 | 0.8152 | 0.6890 | 0.7150 |
78
- | Llama 3.1 Swallow 70B Instruct | 0.4520 | 0.8148 | 0.6834 | 0.4012 | 0.9157 | 0.7855 | 0.8886 | 0.8486 | 0.5823 | 0.7080 |
79
 
80
  ## MT-Bench JA
81
 
 
68
  ||4-shot|4-shot|4-shot|4-shot|4-shot|5-shot|4-shot|3-shot|0-shot||
69
  ||Acc|EMacc|Acc|EMacc|Acc|Acc|EMacc|CoTEMAcc|pass@1||
70
  | Qwen2-72B-Instruct | 0.4360 | 0.7588 | 0.6857 | 0.3913 | 0.9110 | 0.8391 | 0.8499 | 0.2436 | 0.6939 | 0.6455 |
71
+ | Qwen2.5-72B-Instruct | **0.4540** | 0.6764 | **0.7064** | 0.3550 | 0.8895 | **0.8478** | **0.9113** | 0.4027 | 0.6165 | 0.6511 |
72
  | Llama 3 70B Instruct | 0.4400 | 0.7999 | 0.6552 | 0.4024 | 0.9127 | 0.7992 | 0.9052 | 0.8326 | 0.7555 | 0.7225 |
73
+ | Llama 3.1 70B Instruct | 0.4300 | **0.8212** | 0.6621 | 0.3921 | 0.9157 | 0.8213 | 0.8764 | 0.8390 | **0.7915** | **0.7277** |
74
  | Llama 3 Youko 70B Instruct | 0.4500 | 0.7973 | 0.6863 | 0.3914 | 0.9153 | 0.8055 | 0.8923 | 0.7814 | 0.6598 | 0.7088 |
75
  | Llama-3.1-70B-Japanese-Instruct-2407 | 0.4220 | 0.8104 | 0.6481 | 0.3744 | 0.9170 | 0.8071 | 0.8893 | 0.8228 | 0.7463 | 0.7153 |
76
+ | Llama 3 heron brain 70B v0.3 | 0.4460 | 0.8107 | 0.6682 | **0.4085**| 0.9174 | 0.7898 | 0.8772 | 0.7586 | 0.6713 | 0.7053 |
77
+ | Llama 3 Swallow 70B Instruct | 0.4520 | 0.8174 | 0.6758 | 0.4050 | **0.9230** | 0.7883 | 0.8688 | 0.8152 | 0.6890 | 0.7150 |
78
+ | Llama 3.1 Swallow 70B Instruct | 0.4520 | 0.8148 | 0.6834 | 0.4012 | 0.9157 | 0.7855 | 0.8886 | **0.8486** | 0.5823 | 0.7080 |
79
 
80
  ## MT-Bench JA
81