Adds evaluation on another dataset.

README.md

@@ -5,7 +5,15 @@ metrics:
 - accuracy
 model-index:
 - name: sentiment-polish-gpt2-small
-  results: []
+  results:
+  - task:
+      type: text-classification
+    dataset:
+      type: allegro/klej-polemo2-out
+      name: klej-polemo2-out
+    metrics:
+      - type: accuracy
+        value: 98.38%
 license: mit
 language:
 - pl
@@ -39,7 +47,12 @@ Train/test split: 80%/20%
 Datacollator:
 ```py
 from transformers import DataCollatorWithPadding
-data_collator = DataCollatorWithPadding(tokenizer=tokenizer, padding="longest", max_length=128, pad_to_multiple_of=8)
+data_collator = DataCollatorWithPadding(
+  tokenizer=tokenizer,
+  padding="longest",
+  max_length=128,
+  pad_to_multiple_of=8
+)
 ```
 
 ## Training procedure
@@ -77,6 +90,47 @@ The following hyperparameters were used during training:
 | 0.0069        | 9.0   | 29557 | 0.4529          | 0.9622   |
 | 0.0059        | 10.0  | 32840 | 0.4659          | 0.9627   |
 
+### Evaluation
+
+Evaluated on [allegro/klej-polemo2-out](https://huggingface.co/datasets/allegro/klej-polemo2-out) test dataset.
+```py
+from datasets import load_dataset
+from evaluate import evaluator
+
+data = load_dataset("allegro/klej-polemo2-out", split="test").shuffle(seed=42)
+task_evaluator = evaluator("text-classification")
+
+# fix labels
+l = {
+        "__label__meta_zero": 0,
+        "__label__meta_minus_m": 1,
+        "__label__meta_plus_m": 2,
+        "__label__meta_amb": 3
+    }
+def fix_labels(examples):
+    examples["target"] = l[examples["target"]]
+    return examples
+data = data.map(fix_labels)
+
+eval_resutls = task_evaluator.compute(
+    model_or_pipeline="nie3e/sentiment-polish-gpt2-small",
+    data=data,
+    label_mapping={"NEUTRAL": 0, "NEGATIVE": 1, "POSITIVE": 2, "AMBIGUOUS": 3},
+    input_column="sentence",
+    label_column="target"
+)
+
+print(eval_resutls)
+```
+
+```json
+{
+    "accuracy": 0.9838056680161943,
+    "total_time_in_seconds": 5.2441766999982065,
+    "samples_per_second": 94.1997244296076,
+    "latency_in_seconds": 0.010615742307688678
+}
+```
 
 ### Framework versions