Zero-Shot Classification
Transformers
PyTorch
Safetensors
electra
text-classification
Inference Endpoints
saattrupdan commited on
Commit
76e7935
1 Parent(s): 2f91817

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +12 -0
README.md CHANGED
@@ -60,6 +60,18 @@ You can use this model in your scripts as follows:
60
  0.14446501433849335]}
61
  ```
62
 
 
 
 
 
 
 
 
 
 
 
 
 
63
 
64
  ## Training procedure
65
 
 
60
  0.14446501433849335]}
61
  ```
62
 
63
+ ## Performance
64
+
65
+ As Danish is, as far as we are aware, the only Scandinavian language with a gold standard NLI dataset, namely the [DanFEVER dataset](https://aclanthology.org/2021.nodalida-main.pdf#page=439), we report evaluation scores on the test split of that dataset.
66
+
67
+ We report Matthew's Correlation Coefficient (MCC), macro-average F1-score as well as accuracy.
68
+
69
+ | **Model** | **MCC** | **Macro-F1** | **Accuracy** | **Number of Parameters** |
70
+ | :-------- | :------------ | :--------- | :----------- | :----------- |
71
+ | [`alexandrainst/nb-bert-large-nli-scandi`](https://huggingface.co/alexandrainst/nb-bert-large-nli-scandi) | **73.80%** | **58.41%** | **86.98%** | 354M |
72
+ | [`alexandrainst/nb-bert-base-nli-scandi`](https://huggingface.co/alexandrainst/nb-bert-base-nli-scandi) | 62.44% | 55.00% | 80.42% | 178M |
73
+ | `alexandrainst/electra-small-nordic-nli-scandi` (this) | 47.28% | 48.88% | 73.46% | **22M** |
74
+
75
 
76
  ## Training procedure
77