Hugging Face's logo

yaniseuranova
/
setfit-rag-hybrid-search-query-router

Text Classification
setfit
Safetensors
sentence-transformers
xlm-roberta
generated_from_setfit_trainer
Eval Results
Model card Files Community
yaniseuranova commited on
Commit
7656974
•
1 Parent(s): 6d42cc7

Add SetFit model

Browse files
Files changed (5) hide show
  1. README.md +30 -32
  2. config.json +1 -1
  3. config_setfit.json +3 -3
  4. model.safetensors +1 -1
  5. model_head.pkl +2 -2
README.md CHANGED
@@ -48,7 +48,7 @@ The model has been trained using an efficient few-shot learning technique that i
48
  - **Sentence Transformer body:** [sentence-transformers/paraphrase-mpnet-base-v2](https://huggingface.co/sentence-transformers/paraphrase-mpnet-base-v2)
49
  - **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
50
  - **Maximum Sequence Length:** 512 tokens
51
- - **Number of Classes:** 6 classes
52
  <!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
53
  <!-- - **Language:** Unknown -->
54
  <!-- - **License:** Unknown -->
@@ -60,14 +60,12 @@ The model has been trained using an efficient few-shot learning technique that i
60
  - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
61
 
62
  ### Model Labels
63
- | Label | Examples |
64
- |:----------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
65
- | very_semantic_queries | <ul><li>'Quels sont les principes fondamentaux du corps humain?'</li><li>"Comment améliorer l'efficacité énergétique dans les bâtiments?"</li><li>'Combien de calories dans une pomme?'</li></ul> |
66
- | very_lexical | <ul><li>"Quelle est la capitale de l'Italie?"</li><li>"Qui est l'auteur de '1984'?"</li><li>'What is the current unemployment rate in France?'</li></ul> |
67
- | semantic_queries | <ul><li>"Quels sont les avantages de l'apprentissage machine dans le secteur de la santé?"</li><li>'Comment puis-je optimiser les performances de mon site web?'</li><li>'What are the main challenges in cybersecurity?'</li></ul> |
68
- | lexical | <ul><li>'Quel est le numéro de téléphone du service client ou du customer suport?'</li><li>'How can I reset my user password?'</li><li>'What is the zip code for New York?'</li></ul> |
69
- | lexical_queries | <ul><li>'Comment fonctionne la blockchain?'</li></ul> |
70
- | lexical_query | <ul><li>'Who won the Nobel Peace Prize in 2021?'</li></ul> |
71
 
72
  ## Evaluation
73
 
@@ -128,12 +126,12 @@ preds = model("Comment rédiger un bon CV?")
128
  |:-------------|:----|:-------|:----|
129
  | Word count | 4 | 7.1667 | 13 |
130
 
131
- | Label | Training Sample Count |
132
- |:----------------------|:----------------------|
133
- | very_semantic_queries | 13 |
134
- | semantic_queries | 21 |
135
- | lexical_queries | 1 |
136
- | very_lexical | 15 |
137
 
138
  ### Training Hyperparameters
139
  - batch_size: (16, 16)
@@ -156,23 +154,23 @@ preds = model("Comment rédiger un bon CV?")
156
  | Epoch | Step | Training Loss | Validation Loss |
157
  |:-------:|:-------:|:-------------:|:---------------:|
158
  | 0.0060 | 1 | 0.3581 | - |
159
- | 0.2976 | 50 | 0.2142 | - |
160
- | 0.5952 | 100 | 0.1448 | - |
161
- | 0.8929 | 150 | 0.0094 | - |
162
- | 1.0 | 168 | - | 0.0034 |
163
- | 1.1905 | 200 | 0.0014 | - |
164
- | 1.4881 | 250 | 0.0009 | - |
165
- | 1.7857 | 300 | 0.0007 | - |
166
- | 2.0 | 336 | - | 0.0003 |
167
- | 2.0833 | 350 | 0.0012 | - |
168
- | 2.3810 | 400 | 0.0006 | - |
169
- | 2.6786 | 450 | 0.0004 | - |
170
- | 2.9762 | 500 | 0.0005 | - |
171
- | 3.0 | 504 | - | 0.0003 |
172
- | 3.2738 | 550 | 0.0006 | - |
173
- | 3.5714 | 600 | 0.0004 | - |
174
- | 3.8690 | 650 | 0.0003 | - |
175
- | **4.0** | **672** | **-** | **0.0002** |
176
 
177
  * The bold row denotes the saved checkpoint.
178
  ### Framework Versions
 
48
  - **Sentence Transformer body:** [sentence-transformers/paraphrase-mpnet-base-v2](https://huggingface.co/sentence-transformers/paraphrase-mpnet-base-v2)
49
  - **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
50
  - **Maximum Sequence Length:** 512 tokens
51
+ - **Number of Classes:** 4 classes
52
  <!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
53
  <!-- - **Language:** Unknown -->
54
  <!-- - **License:** Unknown -->
 
60
  - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
61
 
62
  ### Model Labels
63
+ | Label | Examples |
64
+ |:--------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
65
+ | very_semantic | <ul><li>'Quels sont les principes fondamentaux du corps humain?'</li><li>"Comment améliorer l'efficacité énergétique dans les bâtiments?"</li><li>'Combien de calories dans une pomme?'</li></ul> |
66
+ | very_lexical | <ul><li>"Quelle est la capitale de l'Italie?"</li><li>"Qui est l'auteur de '1984'?"</li><li>'What is the current unemployment rate in France?'</li></ul> |
67
+ | semantic | <ul><li>"Quels sont les avantages de l'apprentissage machine dans le secteur de la santé?"</li><li>'Comment puis-je optimiser les performances de mon site web?'</li><li>'What are the main challenges in cybersecurity?'</li></ul> |
68
+ | lexical | <ul><li>'Quel est le numéro de téléphone du service client ou du customer suport?'</li><li>'Comment fonctionne la blockchain?'</li><li>'How can I reset my user password?'</li></ul> |
 
 
69
 
70
  ## Evaluation
71
 
 
126
  |:-------------|:----|:-------|:----|
127
  | Word count | 4 | 7.1667 | 13 |
128
 
129
+ | Label | Training Sample Count |
130
+ |:--------------|:----------------------|
131
+ | very_semantic | 13 |
132
+ | semantic | 21 |
133
+ | lexical | 11 |
134
+ | very_lexical | 15 |
135
 
136
  ### Training Hyperparameters
137
  - batch_size: (16, 16)
 
154
  | Epoch | Step | Training Loss | Validation Loss |
155
  |:-------:|:-------:|:-------------:|:---------------:|
156
  | 0.0060 | 1 | 0.3581 | - |
157
+ | 0.3012 | 50 | 0.2063 | - |
158
+ | 0.6024 | 100 | 0.1283 | - |
159
+ | 0.9036 | 150 | 0.0083 | - |
160
+ | 1.0 | 166 | - | 0.003 |
161
+ | 1.2048 | 200 | 0.0014 | - |
162
+ | 1.5060 | 250 | 0.0008 | - |
163
+ | 1.8072 | 300 | 0.0006 | - |
164
+ | 2.0 | 332 | - | 0.0006 |
165
+ | 2.1084 | 350 | 0.0004 | - |
166
+ | 2.4096 | 400 | 0.0003 | - |
167
+ | 2.7108 | 450 | 0.0005 | - |
168
+ | 3.0 | 498 | - | 0.0004 |
169
+ | 3.0120 | 500 | 0.0003 | - |
170
+ | 3.3133 | 550 | 0.0005 | - |
171
+ | 3.6145 | 600 | 0.0003 | - |
172
+ | 3.9157 | 650 | 0.0003 | - |
173
+ | **4.0** | **664** | **-** | **0.0003** |
174
 
175
  * The bold row denotes the saved checkpoint.
176
  ### Framework Versions
config.json CHANGED
@@ -1,5 +1,5 @@
1
  {
2
- "_name_or_path": "checkpoints/step_672",
3
  "architectures": [
4
  "MPNetModel"
5
  ],
 
1
  {
2
+ "_name_or_path": "checkpoints/step_664",
3
  "architectures": [
4
  "MPNetModel"
5
  ],
config_setfit.json CHANGED
@@ -1,8 +1,8 @@
1
  {
2
  "labels": [
3
- "very_semantic_queries",
4
- "semantic_queries",
5
- "lexical_queries",
6
  "very_lexical"
7
  ],
8
  "normalize_embeddings": false
 
1
  {
2
  "labels": [
3
+ "very_semantic",
4
+ "semantic",
5
+ "lexical",
6
  "very_lexical"
7
  ],
8
  "normalize_embeddings": false
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:b7176f55cb144d7f23531b94ed150020322801f7e4507f17192447bbd8d7a497
3
  size 437967672
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1b53398a5e1768704556a85ff27c1d05e5391646db87372f075a3bc0d2145a0b
3
  size 437967672
model_head.pkl CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:70861c361e27e864a51a592de6f9d6069d09dd09a1605f3a871b8061625a56d1
3
- size 38263
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c8319b40498ce63d8e5868820a4e5de714d7af5a0e73cb8705a3d00f2c7f6ee4
3
+ size 25655