Hugging Face's logo

tokyotech-llm
/
Llama-3.1-Swallow-70B-Instruct-v0.1

Text Generation
Transformers
Safetensors
English
Japanese
llama
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
Taishi-N324 commited on
Commit
2b3583b
•
1 Parent(s): 0b77619

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -6
README.md CHANGED
@@ -52,14 +52,14 @@ The website [https://swallow-llm.github.io/](https://swallow-llm.github.io/) pro
52
  | |4-shot|4-shot|4-shot|4-shot|1-shot|4-shot|4-shot|4-shot|5-shot|0-shot| |
53
  | | |EM acc|Char-F1|Char-F1|Char-F1|ROUGE-2|EM acc|BLEU|BLEU|EM acc|pass@1| |
54
  | Qwen2-72B-Instruct | 0.9634 | 0.6268 | 0.5418 | 0.9210 | 0.1644 | 0.7840 | 0.2592 | 0.2327 | 0.7713 | 0.6909 | 0.5955 |
55
- | Qwen2.5-72B-Instruct | 0.9696 | 0.5699 | 0.5811 | 0.7381 | 0.1706 | 0.8360 | 0.2269 | 0.2179 | 0.7899 | 0.6256 | 0.5726 |
56
  | Llama 3 70B Instruct | 0.9419 | 0.6114 | 0.5506 | 0.9164 | 0.1912 | 0.7200 | 0.2708 | 0.2350 | 0.6789 | 0.6610 | 0.5777 |
57
- | Llama 3.1 70B Instruct | 0.9482 | 0.6246 | 0.5781 | 0.9201 | 0.1772 | 0.7440 | 0.2805 | 0.2472 | 0.7323 | 0.6933 | 0.5945 |
58
  | Llama 3 Youko 70B Instruct | 0.9526 | 0.6252 | 0.5853 | 0.9215 | 0.1983 | 0.7400 | 0.2633 | 0.2245 | 0.7170 | 0.6098 | 0.5838 |
59
- | Llama-3.1-70B-Japanese-Instruct-2407 | 0.9562 | 0.6466 | 0.6602 | 0.9187 | 0.1564 | 0.7480 | 0.2901 | 0.2410 | 0.7227 | 0.6274 | 0.5967 |
60
- | Llama 3 heron brain 70B v0.3 | 0.9660 | 0.6643 | 0.6817 | 0.9221 | 0.2611 | 0.7720 | 0.3093 | 0.2578 | 0.7077 | 0.6079 | 0.6150 |
61
- | Llama 3 Swallow 70B Instruct | 0.9607 | 0.6188 | 0.6026 | 0.9236 | 0.1389 | 0.6560 | 0.2724 | 0.2532 | 0.6572 | 0.6000 | 0.5683 |
62
- | Llama 3.1 Swallow 70B Instruct | 0.9598 | 0.6192 | 0.6605 | 0.9235 | 0.1938 | 0.7760 | 0.3123 | 0.2593 | 0.7117 | 0.4713 | 0.5887 |
63
 
64
  ### English tasks
65
 
 
52
  | |4-shot|4-shot|4-shot|4-shot|1-shot|4-shot|4-shot|4-shot|5-shot|0-shot| |
53
  | | |EM acc|Char-F1|Char-F1|Char-F1|ROUGE-2|EM acc|BLEU|BLEU|EM acc|pass@1| |
54
  | Qwen2-72B-Instruct | 0.9634 | 0.6268 | 0.5418 | 0.9210 | 0.1644 | 0.7840 | 0.2592 | 0.2327 | 0.7713 | 0.6909 | 0.5955 |
55
+ | Qwen2.5-72B-Instruct | **0.9696** | 0.5699 | 0.5811 | 0.7381 | 0.1706 | **0.8360** | 0.2269 | 0.2179 | **0.7899** | 0.6256 | 0.5726 |
56
  | Llama 3 70B Instruct | 0.9419 | 0.6114 | 0.5506 | 0.9164 | 0.1912 | 0.7200 | 0.2708 | 0.2350 | 0.6789 | 0.6610 | 0.5777 |
57
+ | Llama 3.1 70B Instruct | 0.9482 | 0.6246 | 0.5781 | 0.9201 | 0.1772 | 0.7440 | 0.2805 | 0.2472 | 0.7323 | **0.6933** | 0.5945 |
58
  | Llama 3 Youko 70B Instruct | 0.9526 | 0.6252 | 0.5853 | 0.9215 | 0.1983 | 0.7400 | 0.2633 | 0.2245 | 0.7170 | 0.6098 | 0.5838 |
59
+ | Llama-3.1-70B-Japanese-Instruct-2407 | 0.9562 | 0.6466 | 0.6602 | 0.9187 | 0.1564 | 0.7480 | 0.2901 | 0.2410 | 0.7227 | 0.6274 | **0.5967** |
60
+ | Llama 3 heron brain 70B v0.3 | 0.9660 | **0.6643** | **0.6817** | 0.9221 | **0.2611** | 0.7720 | 0.3093 | 0.2578 | 0.7077 | 0.6079 | 0.6150 |
61
+ | Llama 3 Swallow 70B Instruct | 0.9607 | 0.6188 | 0.6026 | **0.9236** | 0.1389 | 0.6560 | 0.2724 | 0.2532 | 0.6572 | 0.6000 | 0.5683 |
62
+ | Llama 3.1 Swallow 70B Instruct | 0.9598 | 0.6192 | 0.6605 | 0.9235 | 0.1938 | 0.7760 | **0.3123** | **0.2593** | 0.7117 | 0.4713 | 0.5887 |
63
 
64
  ### English tasks
65