readme: add initial version of model card

#2
by stefan-it - opened
Files changed (1) hide show
  1. README.md +91 -0
README.md ADDED
@@ -0,0 +1,91 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language: de
3
+ license: mit
4
+ tags:
5
+ - flair
6
+ - token-classification
7
+ - sequence-tagger-model
8
+ base_model: deepset/gbert-base
9
+ widget:
10
+ - text: PASt ( KvD ) - Polizeipräsidium Westhessen [ Newsroom ] Wiesbaden ( ots )
11
+ - Am Sonntag , den 27.01.2019 führte die Autobahnpolizei Wiesbaden in Zusammenarbeit
12
+ mit der Präsidialwache in der Zeit von 11:00 - 16:00 Uhr eine Geschwindigkeitsmessung
13
+ in der Baustelle der A66 am Wiesbadener Kreuz durch .
14
+ ---
15
+
16
+ # Fine-tuned Flair Model on German MobIE Dataset with AutoTrain
17
+
18
+ This Flair model was fine-tuned on the
19
+ [German MobIE](https://aclanthology.org/2021.konvens-1.22/)
20
+ NER Dataset using GBERT Base as backbone LM and the 🚀 [AutoTrain](https://github.com/huggingface/autotrain-advanced)
21
+ library.
22
+
23
+ ## Dataset
24
+
25
+ The [German MobIE](https://github.com/DFKI-NLP/MobIE) dataset is a German-language dataset, which is human-annotated
26
+ with 20 coarse- and fine-grained entity types and entity linking information for geographically linkable entities. The
27
+ dataset consists of 3,232 social media texts and traffic reports with 91K tokens, and contains 20.5K annotated
28
+ entities, 13.1K of which are linked to a knowledge base.
29
+
30
+ The following named entities are annotated:
31
+
32
+ * `location-stop`
33
+ * `trigger`
34
+ * `organization-company`
35
+ * `location-city`
36
+ * `location`
37
+ * `event-cause`
38
+ * `location-street`
39
+ * `time`
40
+ * `date`
41
+ * `number`
42
+ * `duration`
43
+ * `organization`
44
+ * `person`
45
+ * `set`
46
+ * `distance`
47
+ * `disaster-type`
48
+ * `money`
49
+ * `org-position`
50
+ * `percent`
51
+
52
+ ## Fine-Tuning
53
+
54
+ The latest [Flair version](https://github.com/flairNLP/flair/tree/42ea3f6854eba04387c38045f160c18bdaac07dc) is used for
55
+ fine-tuning. Additionally, the model is trained with the
56
+ [FLERT (Schweter and Akbik (2020)](https://arxiv.org/abs/2011.06993) approach, because the MobIE dataset thankfully
57
+ comes with document boundary information marker.
58
+
59
+ A hyper-parameter search over the following parameters with 5 different seeds per configuration is performed:
60
+
61
+ * Batch Sizes: [`16`]
62
+ * Learning Rates: [`5e-05`, `3e-05`]
63
+
64
+ All models are trained with the awesome [AutoTrain Advanced](https://github.com/huggingface/autotrain-advanced) from
65
+ Hugging Face. More details can be found in this [repository](https://github.com/stefan-it/autotrain-flair-mobie).
66
+
67
+ ## Results
68
+
69
+ A hyper-parameter search with 5 different seeds per configuration is performed and micro F1-score on development set
70
+ is reported:
71
+
72
+ | Configuration | Seed 1 | Seed 2 | Seed 3 | Seed 4 | Seed 5 | Average |
73
+ |--------------------|-------------|-------------|-------------|-------------|-----------------|-----------------|
74
+ | `bs16-e10-lr5e-05` | [0.8446][1] | [0.8495][2] | [0.8455][3] | [0.8419][4] | [**0.8476**][5] | 0.8458 ± 0.0029 |
75
+ | `bs16-e10-lr3e-05` | [0.8392][6] | [0.8445][7] | [0.8495][8] | [0.8381][9] | [0.8449][10] | 0.8432 ± 0.0046 |
76
+
77
+ [1]: https://hf.co/stefan-it/autotrain-flair-mobie-gbert_base-bs16-e10-lr5e-05-1
78
+ [2]: https://hf.co/stefan-it/autotrain-flair-mobie-gbert_base-bs16-e10-lr5e-05-2
79
+ [3]: https://hf.co/stefan-it/autotrain-flair-mobie-gbert_base-bs16-e10-lr5e-05-3
80
+ [4]: https://hf.co/stefan-it/autotrain-flair-mobie-gbert_base-bs16-e10-lr5e-05-4
81
+ [5]: https://hf.co/stefan-it/autotrain-flair-mobie-gbert_base-bs16-e10-lr5e-05-5
82
+ [6]: https://hf.co/stefan-it/autotrain-flair-mobie-gbert_base-bs16-e10-lr3e-05-1
83
+ [7]: https://hf.co/stefan-it/autotrain-flair-mobie-gbert_base-bs16-e10-lr3e-05-2
84
+ [8]: https://hf.co/stefan-it/autotrain-flair-mobie-gbert_base-bs16-e10-lr3e-05-3
85
+ [9]: https://hf.co/stefan-it/autotrain-flair-mobie-gbert_base-bs16-e10-lr3e-05-4
86
+ [10]: https://hf.co/stefan-it/autotrain-flair-mobie-gbert_base-bs16-e10-lr3e-05-5
87
+
88
+ The result in bold shows the performance of this model.
89
+
90
+ Additionally, the Flair [training log](training.log) and [TensorBoard logs](tensorboard) are also uploaded to the model
91
+ hub.