Update README.md
Browse files
README.md
CHANGED
|
@@ -57,4 +57,16 @@ Note, this is just a POC/test and I don't think it's actually so useful. If you
|
|
| 57 |
>
|
| 58 |
> The Chinese government has denied these allegations, claiming that the camps provide vocational training and education to help combat extremism and terrorism. However, evidence from satellite imagery, leaked documents, and eyewitness accounts suggest otherwise.
|
| 59 |
>
|
| 60 |
-
> Human rights organizations, including Amnesty International and Human Rights Watch, have condemned the Chinese government's actions in Xinjiang and called for an independent investigation into the situation. The United Nations has also expressed concern about the human rights abuses taking place in the region.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 57 |
>
|
| 58 |
> The Chinese government has denied these allegations, claiming that the camps provide vocational training and education to help combat extremism and terrorism. However, evidence from satellite imagery, leaked documents, and eyewitness accounts suggest otherwise.
|
| 59 |
>
|
| 60 |
+
> Human rights organizations, including Amnesty International and Human Rights Watch, have condemned the Chinese government's actions in Xinjiang and called for an independent investigation into the situation. The United Nations has also expressed concern about the human rights abuses taking place in the region.
|
| 61 |
+
|
| 62 |
+
I did manual testing comparison with [cognitivecomputations/dolphin-2.9.2-qwen2-7b](https://huggingface.co/cognitivecomputations/dolphin-2.9.2-qwen2-7b), which was a fine-tune off of the Qwen2 base model and did not notice any similar Chinese alignment issues, so if you're *really* looking for a Qwen2-based model, I'd highly recommend using that instead.
|
| 63 |
+
|
| 64 |
+
On general model performance:
|
| 65 |
+
|
| 66 |
+
| Model | Overall | MATH | BBH | DROP | GSM8k | AGIEval | TriviaQA | MBPP | MMLU | HellaSwag | BoolQ | GPQA | PIQA | OpenBookQA | ARC | CommonsenseQA | SIQA |
|
| 67 |
+
|------------------------------------------------------------------------------------------------|---------|------|------|------|-------|---------|----------|------|------|-----------|-------|------|------|------------|-----|---------------|------|
|
| 68 |
+
| [Llama 3 8B Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct) | 0.4105 | 0.45 | 0.556 | 0.525 | 0.595 | 0.352 | 0.324 | 0.0 | 0.403 | 0.344 | 0.324 | 0.25 | 0.75 | 0.75 | 0.0 | 0.52 | 0.45 |
|
| 69 |
+
| [Qwen 2 7B Instruct](https://huggingface.co/Qwen/Qwen2-7B-Instruct) | 0.4345 | 0.756 | 0.744 | 0.546 | 0.741 | 0.479 | 0.319 | 1.0 | 0.377 | 0.443 | 0.243 | 0.25 | 0.25 | 0.75 | 0.0 | 0.58 | 0.40 |
|
| 70 |
+
| [Qwen 2 7B Instruct deccp](https://huggingface.co/augmxnt/Qwen2-7B-Instruct-deccp) | 0.4395 | 0.9 | 0.738 | 0.575 | 0.786 | 0.479 | 0.312 | 1.0 | 0.372 | 0.443 | 0.243 | 0.25 | 0.25 | 0.75 | 0.0 | 0.58 | 0.40 |
|
| 71 |
+
| [Dolphin 2.9.2 Qwen2 7B](https://huggingface.co/cognitivecomputations/dolphin-2.9.2-qwen2-7b) | 0.4115 | 0.637 | 0.738 | 0.664 | 0.691 | 0.296 | 0.398 | 0.0 | 0.29 | 0.23 | 0.351 | 0.125 | 0.25 | 0.5 | 0.25| 0.26 | 0.55 |
|
| 72 |
+
|