Hugging Face's logo

CATIE-AQ
/
NERmembert-base-4entities

Token Classification
Transformers
TensorBoard
Safetensors
French
camembert
Inference Endpoints
Model card Files Metrics Community
NERmembert-base-4entities / README.md
bourdoiscatie's picture
bourdoiscatie
Update README.md
05ead53 verified
preview code
|
raw
history blame
31.2 kB
---
license: mit
base_model: camembert-base
metrics:
- precision
- recall
- f1
- accuracy
model-index:
- name: NERmembert-base-4entities
results: []
datasets:
- CATIE-AQ/frenchNER_4entities
language:
- fr
widget:
- text: "Assurés de disputer l'Euro 2024 en Allemagne l'été prochain (du 14 juin au 14 juillet) depuis leur victoire aux Pays-Bas, les Bleus ont fait le nécessaire pour avoir des certitudes. Avec six victoires en six matchs officiels et un seul but encaissé, Didier Deschamps a consolidé les acquis de la dernière Coupe du monde. Les joueurs clés sont connus : Kylian Mbappé, Aurélien Tchouameni, Antoine Griezmann, Ibrahima Konaté ou encore Mike Maignan."
library_name: transformers
pipeline_tag: token-classification
co2_eq_emissions: 20
---
# NERmembert-base-4entities
## Model Description
We present **NERmembert-base-4entities**, which is a [CamemBERT base](https://huggingface.co./camembert-base) fine-tuned for the Name Entity Recognition task for the French language on four French NER datasets for 4 entities (LOC, PER, ORG, MISC).
All these datasets were concatenated and cleaned into a single dataset that we called [frenchNER_4entities](https://huggingface.co./datasets/CATIE-AQ/frenchNER_4entities).
There are a total of **384,773** rows, of which **328,757** are for training, **24,131** for validation and **31,885** for testing.
Our methodology is described in a blog post available in [English](https://blog.vaniila.ai/en/NER_en/) or [French](https://blog.vaniila.ai/NER/).
## Dataset
The dataset used is [frenchNER_4entities](https://huggingface.co./datasets/CATIE-AQ/frenchNER_4entities), which represents ~385k sentences labeled in 4 categories:
| Label | Examples |
|:------|:-----------------------------------------------------------|
| PER | "La Bruyère", "Gaspard de Coligny", "Wittgenstein" |
| ORG | "UTBM", "American Airlines", "id Software" |
| LOC | "République du Cap-Vert", "Créteil", "Bordeaux" |
| MISC | "Wolfenstein 3D", "Révolution française", "Coupe du monde" |
The distribution of the entities is as follows:
<table>
<thead>
<tr>
<th><br>Splits</th>
<th><br>O</th>
<th><br>PER</th>
<th><br>LOC</th>
<th><br>ORG</th>
<th><br>MISC</th>
</tr>
</thead>
<tbody>
<td><br>train</td>
<td><br><b>7,539,692</b></td>
<td><br><b>307,144</b></td>
<td><br><b>286,746</b></td>
<td><br><b>127,089</b></td>
<td><br><b>799,494</b></td>
</tr>
<tr>
<td><br>validation</td>
<td><br><b>544,580</b></td>
<td><br><b>24,034</b></td>
<td><br><b>21,585</b></td>
<td><br><b>5,927</b></td>
<td><br><b>18,221</b></td>
</tr>
<tr>
<td><br>test</td>
<td><br><b>720,623</b></td>
<td><br><b>32,870</b></td>
<td><br><b>29,683</b></td>
<td><br><b>7,911</b></td>
<td><br><b>21,760</b></td>
</tr>
</tbody>
</table>
## Evaluation results
The evaluation was carried out using the [**evaluate**](https://pypi.org/project/evaluate/) python package.
### frenchNER_4entities
For space reasons, we show only the F1 of the different models. You can see the full results below the table.
<table>
<thead>
<tr>
<th><br>Model</th>
<th><br>PER</th>
<th><br>LOC</th>
<th><br>ORG</th>
<th><br>MISC</th>
</tr>
</thead>
<tbody>
<tr>
<td rowspan="1"><br><a href="https://hf.co/Jean-Baptiste/camembert-ner">Jean-Baptiste/camembert-ner</a></td>
<td><br>0.971</td>
<td><br>0.947</td>
<td><br>0.902</td>
<td><br>0.663</td>
</tr>
<tr>
<td rowspan="1"><br><a href="https://hf/cmarkea/distilcamembert-base-ner">cmarkea/distilcamembert-base-ner</a></td>
<td><br>0.974</td>
<td><br>0.948</td>
<td><br>0.892</td>
<td><br>0.658</td>
</tr>
<tr>
<td rowspan="1"><br><a href="https://hf/CATIE-AQ/NERmembert-base-3entities">NERmembert-base-3entities</a></td>
<td><br>A</td>
<td><br>B</td>
<td><br>C</td>
<td><br>0</td>
</tr>
<tr>
<td rowspan="1"><br>NERmembert-base-4entities (this model)</td>
<td><br><b>0.978</b></td>
<td><br><b>0.958</b></td>
<td><br><b>0.903</b></td>
<td><br><b>0.814</b></td>
</tr>
<tr>
<td rowspan="1"><br><a href="https://hf/CATIE-AQ/NERmembert-large-4entities">NERmembert-large-4entities</a></td>
<td><br>A</td>
<td><br>B</td>
<td><br>C</td>
<td><br>D</td>
</tr>
</tbody>
</table>
<details>
<summary>Full results</summary>
<table>
<thead>
<tr>
<th><br>Model</th>
<th><br>Metrics</th>
<th><br>PER</th>
<th><br>LOC</th>
<th><br>ORG</th>
<th><br>MISC</th>
<th><br>O</th>
<th><br>Overall</th>
</tr>
</thead>
<tbody>
<tr>
<td rowspan="3"><br><a href="https://hf.co/Jean-Baptiste/camembert-ner">Jean-Baptiste/camembert-ner</a></td>
<td><br>Precision</td>
<td><br>0.952</td>
<td><br>0.924</td>
<td><br>0.870</td>
<td><br>0.845</td>
<td><br>0.986</td>
<td><br>0.976</td>
</tr>
<tr>
<td><br>Recall</td>
<td><br>0.990</td>
<td><br>0.972</td>
<td><br>0.938</td>
<td><br>0.546</td>
<td><br>0.992</td>
<td><br>0.976</td>
</tr>
<tr>
<td>F1</td>
<td><br>0.971</td>
<td><br>0.947</td>
<td><br>0.902</td>
<td><br>0.663</td>
<td><br>0.989</td>
<td><br>0.976</td>
</tr>
<tr>
<td rowspan="3"><br><a href="https://hf/cmarkea/distilcamembert-base-ner">cmarkea/distilcamembert-base-ner</a></td>
<td><br>Precision</td>
<td><br>0.962</td>
<td><br>0.933</td>
<td><br>0.857</td>
<td><br>0.830</td>
<td><br>0.985</td>
<td><br>0.976</td>
</tr>
<tr>
<td><br>Recall</td>
<td><br>0.987</td>
<td><br>0.963</td>
<td><br>0.930</td>
<td><br>0.545</td>
<td><br>0.993</td>
<td><br>0.976</td>
</tr>
<tr>
<td>F1</td>
<td><br>0.974</td>
<td><br>0.948</td>
<td><br>0.892</td>
<td><br>0.658</td>
<td><br>0.989</td>
<td><br>0.976</td>
</tr>
<tr>
<td rowspan="3"><br><a href="https://hf/CATIE-AQ/NERmembert-base-3entities">NERmembert-base-3entities</a></td>
<td><br>Precision</td>
<td><br>A</td>
<td><br>B</td>
<td><br>C</td>
<td><br>0</td>
<td><br>X</td>
<td><br>X</td>
</tr>
<tr>
<td><br>Recall</td>
<td><br>A</td>
<td><br>B</td>
<td><br>C</td>
<td><br>0</td>
<td><br>X</td>
<td><br>X</td>
</tr>
<tr>
<td>F1</td>
<td><br>A</td>
<td><br>B</td>
<td><br>C</td>
<td><br>0</td>
<td><br>X</td>
<td><br>X</td>
</tr>
<tr>
<td rowspan="3"><br>NERmembert-base-4entities (this model)</td>
<td><br>Precision</td>
<td><br>0.973</td>
<td><br>0.951</td>
<td><br>0.888</td>
<td><br>0.850</td>
<td><br>0.993</td>
<td><br>0.984</td>
</tr>
<tr>
<td><br>Recall</td>
<td><br>0.983</td>
<td><br>0.964</td>
<td><br>0.918</td>
<td><br>0.781</td>
<td><br>0.993</td>
<td><br>0.984</td>
</tr>
<tr>
<td>F1</td>
<td><br><b>0.978</b></td>
<td><br><b>0.958</b></td>
<td><br><b>0.903</b></td>
<td><br><b>0.814</b></td>
<td><br><b>0.993</b></td>
<td><br><b>0.984</b></td>
</tr>
<tr>
<td rowspan="3"><br><a href="https://hf/CATIE-AQ/NERmembert-large-4entities">NERmembert-large-4entities</a></td>
<td><br>Precision</td>
<td><br>A</td>
<td><br>B</td>
<td><br>C</td>
<td><br>D</td>
<td><br>E</td>
<td><br>F</td>
</tr>
<tr>
<td><br>Recall</td>
<td><br>A</td>
<td><br>B</td>
<td><br>C</td>
<td><br>D</td>
<td><br>E</td>
<td><br>F</td>
</tr>
<tr>
<td>F1</td>
<td><br>A</td>
<td><br>B</td>
<td><br>C</td>
<td><br>D</td>
<td><br>E</td>
<td><br>F</td>
</tr>
</tbody>
</table>
</details>
In detail:
### multiconer
For space reasons, we show only the F1 of the different models. You can see the full results below the table.
<table>
<thead>
<tr>
<th><br>Model</th>
<th><br>PER</th>
<th><br>LOC</th>
<th><br>ORG</th>
<th><br>MISC</th>
</tr>
</thead>
<tbody>
<tr>
<td rowspan="1"><br><a href="https://hf.co/Jean-Baptiste/camembert-ner">Jean-Baptiste/camembert-ner</a></td>
<td><br>0.940</td>
<td><br>0.761</td>
<td><br>0.723</td>
<td><br>0.560</td>
</tr>
<tr>
<td rowspan="1"><br><a href="https://hf/cmarkea/distilcamembert-base-ner">cmarkea/distilcamembert-base-ner</a></td>
<td><br>0.921</td>
<td><br>0.748</td>
<td><br>0.694</td>
<td><br>0.530</td>
</tr>
<tr>
<td rowspan="1"><br><a href="https://hf/CATIE-AQ/NERmembert-base-3entities">NERmembert-base-3entities</a></td>
<td><br>A</td>
<td><br>B</td>
<td><br>C</td>
<td><br>0</td>
</tr>
<tr>
<td rowspan="1"><br>NERmembert-base-4entities (this model)</td>
<td><br><b>0.960</b></td>
<td><br><b>0.890</b></td>
<td><br><b>0.867</b></td>
<td><br><b>0.852</b></td>
</tr>
<tr>
<td rowspan="1"><br><a href="https://hf/CATIE-AQ/NERmembert-large-4entities">NERmembert-large-4entities</a></td>
<td><br>A</td>
<td><br>B</td>
<td><br>C</td>
<td><br>D</td>
</tr>
</tbody>
</table>
<details>
<summary>Full results</summary>
<table>
<thead>
<tr>
<th><br>Model</th>
<th><br>Metrics</th>
<th><br>PER</th>
<th><br>LOC</th>
<th><br>ORG</th>
<th><br>MISC</th>
<th><br>O</th>
<th><br>Overall</th>
</tr>
</thead>
<tbody>
<tr>
<td rowspan="3"><br><a href="https://hf.co/Jean-Baptiste/camembert-ner">Jean-Baptiste/camembert-ner</a></td>
<td><br>Precision</td>
<td><br>0.908</td>
<td><br>0.717</td>
<td><br>0.753</td>
<td><br>0.620</td>
<td><br>0.936</td>
<td><br>0.889</td>
</tr>
<tr>
<td><br>Recall</td>
<td><br>0.975</td>
<td><br>0.811</td>
<td><br>0.696</td>
<td><br>0.511</td>
<td><br>0.938</td>
<td><br>0.889</td>
</tr>
<tr>
<td>F1</td>
<td><br>0.940</td>
<td><br>0.761</td>
<td><br>0.723</td>
<td><br>0.560</td>
<td><br>0.937</td>
<td><br>0.889</td>
</tr>
<tr>
<td rowspan="3"><br><a href="https://hf/cmarkea/distilcamembert-base-ner">cmarkea/distilcamembert-base-ner</a></td>
<td><br>Precision</td>
<td><br>0.885</td>
<td><br>0.738</td>
<td><br>0.737</td>
<td><br>0.589</td>
<td><br>0.928</td>
<td><br>0.881</td>
</tr>
<tr>
<td><br>Recall</td>
<td><br>0.960</td>
<td><br>0.759</td>
<td><br>0.655</td>
<td><br>0.482</td>
<td><br>0.939</td>
<td><br>0.881</td>
</tr>
<tr>
<td>F1</td>
<td><br>0.921</td>
<td><br>0.748</td>
<td><br>0.694</td>
<td><br>0.530</td>
<td><br>0.934</td>
<td><br>0.881</td>
</tr>
<tr>
<td rowspan="3"><br><a href="https://hf/CATIE-AQ/NERmembert-base-3entities">NERmembert-base-3entities</a></td>
<td><br>Precision</td>
<td><br>A</td>
<td><br>B</td>
<td><br>C</td>
<td><br>0</td>
<td><br>X</td>
<td><br>X</td>
</tr>
<tr>
<td><br>Recall</td>
<td><br>A</td>
<td><br>B</td>
<td><br>C</td>
<td><br>0</td>
<td><br>X</td>
<td><br>X</td>
</tr>
<tr>
<td>F1</td>
<td><br>A</td>
<td><br>B</td>
<td><br>C</td>
<td><br>0</td>
<td><br>X</td>
<td><br>X</td>
</tr>
<tr>
<td rowspan="3"><br>NERmembert-base-4entities (this model)</td>
<td><br>Precision</td>
<td><br>0.954</td>
<td><br>0.893</td>
<td><br>0.851</td>
<td><br>0.849</td>
<td><br>0.979</td>
<td><br>0.954</td>
</tr>
<tr>
<td><br>Recall</td>
<td><br>0.967</td>
<td><br>0.887</td>
<td><br>0.883</td>
<td><br>0.855</td>
<td><br>0.974</td>
<td><br>0.954</td>
</tr>
<tr>
<td>F1</td>
<td><br><b>0.960</b></td>
<td><br><b>0.890</b></td>
<td><br><b>0.867</b></td>
<td><br><b>0.852</b></td>
<td><br><b>0.977</b></td>
<td><br><b>0.954</b></td>
</tr>
<tr>
<td rowspan="3"><br><a href="https://hf/CATIE-AQ/NERmembert-large-4entities">NERmembert-large-4entities</a></td>
<td><br>Precision</td>
<td><br>A</td>
<td><br>B</td>
<td><br>C</td>
<td><br>D</td>
<td><br>E</td>
<td><br>F</td>
</tr>
<tr>
<td><br>Recall</td>
<td><br>A</td>
<td><br>B</td>
<td><br>C</td>
<td><br>D</td>
<td><br>E</td>
<td><br>F</td>
</tr>
<tr>
<td>F1</td>
<td><br>A</td>
<td><br>B</td>
<td><br>C</td>
<td><br>D</td>
<td><br>E</td>
<td><br>F</td>
</tr>
</tbody>
</table>
</details>
### multinerd
For space reasons, we show only the F1 of the different models. You can see the full results below the table.
<table>
<thead>
<tr>
<th><br>Model</th>
<th><br>PER</th>
<th><br>LOC</th>
<th><br>ORG</th>
<th><br>MISC</th>
</tr>
</thead>
<tbody>
<tr>
<td rowspan="1"><br><a href="https://hf.co/Jean-Baptiste/camembert-ner">Jean-Baptiste/camembert-ner</a></td>
<td><br>0.962</td>
<td><br>0.934</td>
<td><br>0.888</td>
<td><br>0.419</td>
</tr>
<tr>
<td rowspan="1"><br><a href="https://hf/cmarkea/distilcamembert-base-ner">cmarkea/distilcamembert-base-ner</a></td>
<td><br>0.972</td>
<td><br>0.938</td>
<td><br>0.884</td>
<td><br>0.430</td>
</tr>
<tr>
<td rowspan="1"><br><a href="https://hf/CATIE-AQ/NERmembert-base-3entities">NERmembert-base-3entities</a></td>
<td><br>A</td>
<td><br>B</td>
<td><br>C</td>
<td><br>0</td>
</tr>
<tr>
<td rowspan="1"><br>NERmembert-base-4entities (this model)</td>
<td><br><b>0.985</b></td>
<td><br><b>0.973</b></td>
<td><br><b>0.938</b></td>
<td><br><b>0.770</b></td>
</tr>
<tr>
<td rowspan="1"><br><a href="https://hf/CATIE-AQ/NERmembert-large-4entities">NERmembert-large-4entities</a></td>
<td><br>A</td>
<td><br>B</td>
<td><br>C</td>
<td><br>D</td>
</tr>
</tbody>
</table>
<details>
<summary>Full results</summary>
<table>
<thead>
<tr>
<th><br>Model</th>
<th><br>Metrics</th>
<th><br>PER</th>
<th><br>LOC</th>
<th><br>ORG</th>
<th><br>MISC</th>
<th><br>O</th>
<th><br>Overall</th>
</tr>
</thead>
<tbody>
<tr>
<td rowspan="3"><br><a href="https://hf.co/Jean-Baptiste/camembert-ner">Jean-Baptiste/camembert-ner</a></td>
<td><br>Precision</td>
<td><br>0.931</td>
<td><br>0.893</td>
<td><br>0.827</td>
<td><br>0.725</td>
<td><br>0.979</td>
<td><br>0.966</td>
</tr>
<tr>
<td><br>Recall</td>
<td><br>0.994</td>
<td><br>0.980</td>
<td><br>0.959</td>
<td><br>0.295</td>
<td><br>0.990</td>
<td><br>0.966</td>
</tr>
<tr>
<td>F1</td>
<td><br>0.962</td>
<td><br>0.934</td>
<td><br>0.888</td>
<td><br>0.419</td>
<td><br>0.984</td>
<td><br>0.966</td>
</tr>
<tr>
<td rowspan="3"><br><a href="https://hf/cmarkea/distilcamembert-base-ner">cmarkea/distilcamembert-base-ner</a></td>
<td><br>Precision</td>
<td><br>0.954</td>
<td><br>0.908</td>
<td><br>0.817</td>
<td><br>0.705</td>
<td><br>0.977</td>
<td><br>0.967</td>
</tr>
<tr>
<td><br>Recall</td>
<td><br>0.991</td>
<td><br>0.969</td>
<td><br>0.963</td>
<td><br>0.310</td>
<td><br>0.990</td>
<td><br>0.967</td>
</tr>
<tr>
<td>F1</td>
<td><br>0.972</td>
<td><br>0.938</td>
<td><br>0.884</td>
<td><br>0.430</td>
<td><br>0.984</td>
<td><br>0.967</td>
</tr>
<tr>
<td rowspan="3"><br><a href="https://hf/CATIE-AQ/NERmembert-base-3entities">NERmembert-base-3entities</a></td>
<td><br>Precision</td>
<td><br>A</td>
<td><br>B</td>
<td><br>C</td>
<td><br>0</td>
<td><br>X</td>
<td><br>X</td>
</tr>
<tr>
<td><br>Recall</td>
<td><br>A</td>
<td><br>B</td>
<td><br>C</td>
<td><br>0</td>
<td><br>X</td>
<td><br>X</td>
</tr>
<tr>
<td>F1</td>
<td><br>A</td>
<td><br>B</td>
<td><br>C</td>
<td><br>0</td>
<td><br>X</td>
<td><br>X</td>
</tr>
<tr>
<td rowspan="3"><br>NERmembert-base-4entities (this model)</td>
<td><br>Precision</td>
<td><br>0.976</td>
<td><br>0.961</td>
<td><br>0.91</td>
<td><br>0.829</td>
<td><br>0.991</td>
<td><br>0.983</td>
</tr>
<tr>
<td><br>Recall</td>
<td><br>0.994</td>
<td><br>0.985</td>
<td><br>0.967</td>
<td><br>0.719</td>
<td><br>0.993</td>
<td><br>0.983</td>
</tr>
<tr>
<td>F1</td>
<td><br><b>0.985</b></td>
<td><br><b>0.973</b></td>
<td><br><b>0.938</b></td>
<td><br><b>0.770</b></td>
<td><br><b>0.992</b></td>
<td><br><b>0.983</b></td>
</tr>
<tr>
<td rowspan="3"><br><a href="https://hf/CATIE-AQ/NERmembert-large-4entities">NERmembert-large-4entities</a></td>
<td><br>Precision</td>
<td><br>A</td>
<td><br>B</td>
<td><br>C</td>
<td><br>D</td>
<td><br>E</td>
<td><br>F</td>
</tr>
<tr>
<td><br>Recall</td>
<td><br>A</td>
<td><br>B</td>
<td><br>C</td>
<td><br>D</td>
<td><br>E</td>
<td><br>F</td>
</tr>
<tr>
<td>F1</td>
<td><br>A</td>
<td><br>B</td>
<td><br>C</td>
<td><br>D</td>
<td><br>E</td>
<td><br>F</td>
</tr>
</tbody>
</table>
</details>
### wikiner
For space reasons, we show only the F1 of the different models. You can see the full results below the table.
<table>
<thead>
<tr>
<th><br>Model</th>
<th><br>PER</th>
<th><br>LOC</th>
<th><br>ORG</th>
<th><br>MISC</th>
</tr>
</thead>
<tbody>
<tr>
<td rowspan="1"><br><a href="https://hf.co/Jean-Baptiste/camembert-ner">Jean-Baptiste/camembert-ner</a></td>
<td><br><b>0.986</b></td>
<td><br><b>0.966</b></td>
<td><br><b>0.938</b></td>
<td><br><b>0.938</b></td>
</tr>
<tr>
<td rowspan="1"><br><a href="https://hf/cmarkea/distilcamembert-base-ner">cmarkea/distilcamembert-base-ner</a></td>
<td><br>0.983</td>
<td><br>0.964</td>
<td><br>0.925</td>
<td><br>0.926</td>
</tr>
<tr>
<td rowspan="1"><br><a href="https://hf/CATIE-AQ/NERmembert-base-3entities">NERmembert-base-3entities</a></td>
<td><br>A</td>
<td><br>B</td>
<td><br>C</td>
<td><br>0</td>
</tr>
<tr>
<td rowspan="1"><br>NERmembert-base-4entities (this model)</td>
<td><br>0.970</td>
<td><br>0.945</td>
<td><br>0.876</td>
<td><br>0.872</td>
</tr>
<tr>
<td rowspan="1"><br><a href="https://hf/CATIE-AQ/NERmembert-large-4entities">NERmembert-large-4entities</a></td>
<td><br>A</td>
<td><br>B</td>
<td><br>C</td>
<td><br>D</td>
</tr>
</tbody>
</table>
<details>
<summary>Full results</summary>
<table>
<thead>
<tr>
<th><br>Model</th>
<th><br>Metrics</th>
<th><br>PER</th>
<th><br>LOC</th>
<th><br>ORG</th>
<th><br>MISC</th>
<th><br>O</th>
<th><br>Overall</th>
</tr>
</thead>
<tbody>
<tr>
<td rowspan="3"><br><a href="https://hf.co/Jean-Baptiste/camembert-ner">Jean-Baptiste/camembert-ner</a></td>
<td><br>Precision</td>
<td><br>0.986</td>
<td><br>0.962</td>
<td><br>0.925</td>
<td><br>0.943</td>
<td><br>0.998</td>
<td><br>0.992</td>
</tr>
<tr>
<td><br>Recall</td>
<td><br>0.987</td>
<td><br>0.969</td>
<td><br>0.951</td>
<td><br>0.933</td>
<td><br>0.997</td>
<td><br>0.992</td>
</tr>
<tr>
<td>F1</td>
<td><br><b>0.986</b></td>
<td><br><b>0.966</b></td>
<td><br><b>0.938</b></td>
<td><br><b>0.938</b></td>
<td><br><b>0.998</b></td>
<td><br><b>0.992</b></td>
</tr>
<tr>
<td rowspan="3"><br><a href="https://hf/cmarkea/distilcamembert-base-ner">cmarkea/distilcamembert-base-ner</a></td>
<td><br>Precision</td>
<td><br>0.982</td>
<td><br>0.964</td>
<td><br>0.910</td>
<td><br>0.942</td>
<td><br>0.997</td>
<td><br>0.991</td>
</tr>
<tr>
<td><br>Recall</td>
<td><br>0.985</td>
<td><br>0.963</td>
<td><br>0.940</td>
<td><br>0.910</td>
<td><br>0.998</td>
<td><br>0.991</td>
</tr>
<tr>
<td>F1</td>
<td><br>0.983</td>
<td><br>0.964</td>
<td><br>0.925</td>
<td><br>0.926</td>
<td><br>0.997</td>
<td><br>0.991</td>
</tr>
<tr>
<td rowspan="3"><br><a href="https://hf/CATIE-AQ/NERmembert-base-3entities">NERmembert-base-3entities</a></td>
<td><br>Precision</td>
<td><br>A</td>
<td><br>B</td>
<td><br>C</td>
<td><br>0</td>
<td><br>X</td>
<td><br>X</td>
</tr>
<tr>
<td><br>Recall</td>
<td><br>A</td>
<td><br>B</td>
<td><br>C</td>
<td><br>0</td>
<td><br>X</td>
<td><br>X</td>
</tr>
<tr>
<td>F1</td>
<td><br>A</td>
<td><br>B</td>
<td><br>C</td>
<td><br>0</td>
<td><br>X</td>
<td><br>X</td>
</tr>
<tr>
<td rowspan="3"><br>NERmembert-base-4entities (this model)</td>
<td><br>Precision</td>
<td><br>0.970</td>
<td><br>0.944</td>
<td><br>0.872</td>
<td><br>0.878</td>
<td><br>0.996</td>
<td><br>0.986</td>
</tr>
<tr>
<td><br>Recall</td>
<td><br>0.969</td>
<td><br>0.947</td>
<td><br>0.880</td>
<td><br>0.866</td>
<td><br>0.996</td>
<td><br>0.986</td>
</tr>
<tr>
<td>F1</td>
<td><br>0.970</td>
<td><br>0.945</td>
<td><br>0.876</td>
<td><br>0.872</td>
<td><br>0.996</td>
<td><br>0.986</td>
</tr>
<tr>
<td rowspan="3"><br><a href="https://hf/CATIE-AQ/NERmembert-large-4entities">NERmembert-large-4entities</a></td>
<td><br>Precision</td>
<td><br>A</td>
<td><br>B</td>
<td><br>C</td>
<td><br>D</td>
<td><br>E</td>
<td><br>F</td>
</tr>
<tr>
<td><br>Recall</td>
<td><br>A</td>
<td><br>B</td>
<td><br>C</td>
<td><br>D</td>
<td><br>E</td>
<td><br>F</td>
</tr>
<tr>
<td>F1</td>
<td><br>A</td>
<td><br>B</td>
<td><br>C</td>
<td><br>D</td>
<td><br>E</td>
<td><br>F</td>
</tr>
</tbody>
</table>
</details>
## Usage
### Code
```python
from transformers import pipeline
ner = pipeline('question-answering', model='CATIE-AQ/NERmembert-base-4entities', tokenizer='CATIE-AQ/NERmembert-base-4entities', aggregation_strategy="simple")
results = ner(
"Assurés de disputer l'Euro 2024 en Allemagne l'été prochain (du 14 juin au 14 juillet) depuis leur victoire aux Pays-Bas, les Bleus ont fait le nécessaire pour avoir des certitudes. Avec six victoires en six matchs officiels et un seul but encaissé, Didier Deschamps a consolidé les acquis de la dernière Coupe du monde. Les joueurs clés sont connus : Kylian Mbappé, Aurélien Tchouameni, Antoine Griezmann, Ibrahima Konaté ou encore Mike Maignan."
)
print(result)
```python
[{'entity_group': 'MISC',
'score': 0.9404951632022858,
'word': 'Euro 2024',
'start': 22,
'end': 31},
{'entity_group': 'LOC',
'score': 0.96980727,
'word': 'Allemagne',
'start': 35,
'end': 44},
{'entity_group': 'LOC',
'score': 0.8612850904464722,
'word': 'Pays-Bas',
'start': 112,
'end': 120},
{'entity_group': 'ORG',
'score': 0.8148028254508972,
'word': 'les Bleus',
'start': 122,
'end': 131},
{'entity_group': 'PER',
'score': 0.9994482398033142,
'word': 'Didier Deschamps',
'start': 250,
'end': 266},
{'entity_group': 'MISC',
'score': 0.84807388484478,
'word': 'dernière Coupe du monde',
'start': 296,
'end': 319},
{'entity_group': 'PER',
'score': 0.9996860176324844,
'word': 'Kylian Mbappé',
'start': 352,
'end': 365},
{'entity_group': 'PER',
'score': 0.9996881932020187,
'word': 'Aurélien Tchouameni',
'start': 367,
'end': 386},
{'entity_group': 'PER',
'score': 0.9996924996376038,
'word': 'Antoine Griezmann',
'start': 388,
'end': 405},
{'entity_group': 'PER',
'score': 0.9996860027313232,
'word': 'Ibrahima Konaté',
'start': 407,
'end': 422},
{'entity_group': 'PER',
'score': 0.9996623992919922,
'word': 'Mike Maignan',
'start': 433,
'end': 445}]
```
### Try it through Space
A Space has been created to test the model. It is available [here](https://huggingface.co./spaces/CATIE-AQ/NERmembert).
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3
### Training results
| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1 | Accuracy |
|:-------------:|:-----:|:------:|:---------------:|:---------:|:------:|:------:|:--------:|
| 0.0407 | 1.0 | 41095 | 0.0547 | 0.9816 | 0.9816 | 0.9816 | 0.9816 |
| 0.0242 | 2.0 | 82190 | 0.0488 | 0.9843 | 0.9843 | 0.9843 | 0.9843 |
| 0.018 | 3.0 | 123285 | 0.0542 | 0.9844 | 0.9844 | 0.9844 | 0.9844 |
### Framework versions
- Transformers 4.36.2
- Pytorch 2.1.2
- Datasets 2.16.1
- Tokenizers 0.15.0
## Environmental Impact
*Carbon emissions were estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700). The hardware, runtime, cloud provider, and compute region were utilized to estimate the carbon impact.*
- **Hardware Type:** A100 PCIe 40/80GB
- **Hours used:** 1h45min
- **Cloud Provider:** Private Infrastructure
- **Carbon Efficiency (kg/kWh):** 0.046 (estimated from [electricitymaps](https://app.electricitymaps.com/zone/FR) for the day of January 4, 2024.)
- **Carbon Emitted** *(Power consumption x Time x Carbon produced based on location of power grid)*: 0.02 kg eq. CO2
## Citations
### Camembert-frenchNER_4entities
```
TODO
```
### multiconer
> @inproceedings{multiconer2-report,
title={{SemEval-2023 Task 2: Fine-grained Multilingual Named Entity Recognition (MultiCoNER 2)}},
author={Fetahu, Besnik and Kar, Sudipta and Chen, Zhiyu and Rokhlenko, Oleg and Malmasi, Shervin},
booktitle={Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023)},
year={2023},
publisher={Association for Computational Linguistics}}
> @article{multiconer2-data,
title={{MultiCoNER v2: a Large Multilingual dataset for Fine-grained and Noisy Named Entity Recognition}},
author={Fetahu, Besnik and Chen, Zhiyu and Kar, Sudipta and Rokhlenko, Oleg and Malmasi, Shervin},
year={2023}}
### multinerd
> @inproceedings{tedeschi-navigli-2022-multinerd,
title = "{M}ulti{NERD}: A Multilingual, Multi-Genre and Fine-Grained Dataset for Named Entity Recognition (and Disambiguation)",
author = "Tedeschi, Simone and Navigli, Roberto",
booktitle = "Findings of the Association for Computational Linguistics: NAACL 2022",
month = jul,
year = "2022",
address = "Seattle, United States",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2022.findings-naacl.60",
doi = "10.18653/v1/2022.findings-naacl.60",
pages = "801--812"}
### pii-masking-200k
> @misc {ai4privacy_2023,
author = { {ai4Privacy} },
title = { pii-masking-200k (Revision 1d4c0a1) },
year = 2023,
url = { https://huggingface.co./datasets/ai4privacy/pii-masking-200k },
doi = { 10.57967/hf/1532 },
publisher = { Hugging Face }}
### wikiner
> @article{NOTHMAN2013151,
title = {Learning multilingual named entity recognition from Wikipedia},
journal = {Artificial Intelligence},
volume = {194},
pages = {151-175},
year = {2013},
note = {Artificial Intelligence, Wikipedia and Semi-Structured Resources},
issn = {0004-3702},
doi = {https://doi.org/10.1016/j.artint.2012.03.006},
url = {https://www.sciencedirect.com/science/article/pii/S0004370212000276},
author = {Joel Nothman and Nicky Ringland and Will Radford and Tara Murphy and James R. Curran}}
### frenchNER_4entities
```
TODO
```
### CamemBERT
> @inproceedings{martin2020camembert,
title={CamemBERT: a Tasty French Language Model},
author={Martin, Louis and Muller, Benjamin and Su{\'a}rez, Pedro Javier Ortiz and Dupont, Yoann and Romary, Laurent and de la Clergerie, {\'E}ric Villemonte and Seddah, Djam{\'e} and Sagot, Beno{\^\i}t},
booktitle={Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics},
year={2020}}
## License
[cc-by-4.0](https://creativecommons.org/licenses/by/4.0/deed.en)