Hugging Face's logo

yaniseuranova
/
setfit-rag-hybrid-search-query-router

Text Classification
setfit
Safetensors
sentence-transformers
xlm-roberta
generated_from_setfit_trainer
Eval Results
Model card Files Community
yaniseuranova commited on
Commit
aff388a
•
1 Parent(s): c5e3f1d

Add SetFit model

Browse files
Files changed (5) hide show
  1. README.md +36 -82
  2. config.json +1 -1
  3. config_setfit.json +1 -3
  4. model.safetensors +1 -1
  5. model_head.pkl +2 -2
README.md CHANGED
@@ -9,11 +9,17 @@ base_model: sentence-transformers/all-mpnet-base-v2
9
  metrics:
10
  - accuracy
11
  widget:
12
- - text: Quels sont les enjeux éthiques des algorithmes de décision automatisés?
13
- - text: Who is the founder of Tesla Motors?
14
- - text: How do I create a new email account on Gmail?
15
- - text: How can we use artificial intelligence to improve mental health diagnosis?
16
- - text: What is the definition of a database management system?
 
 
 
 
 
 
17
  pipeline_tag: text-classification
18
  inference: true
19
  model-index:
@@ -48,7 +54,7 @@ The model has been trained using an efficient few-shot learning technique that i
48
  - **Sentence Transformer body:** [sentence-transformers/all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2)
49
  - **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
50
  - **Maximum Sequence Length:** 384 tokens
51
- - **Number of Classes:** 4 classes
52
  <!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
53
  <!-- - **Language:** Unknown -->
54
  <!-- - **License:** Unknown -->
@@ -60,12 +66,10 @@ The model has been trained using an efficient few-shot learning technique that i
60
  - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
61
 
62
  ### Model Labels
63
- | Label | Examples |
64
- |:--------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
65
- | very_semantic | <ul><li>'Quels sont les principes fondamentaux du corps humain?'</li><li>"Comment améliorer l'efficacité énergétique dans les bâtiments?"</li><li>'Combien de calories dans une pomme?'</li></ul> |
66
- | very_lexical | <ul><li>"Quelle est la capitale de l'Italie?"</li><li>"Qui est l'auteur de '1984'?"</li><li>'What is the current unemployment rate in France?'</li></ul> |
67
- | semantic | <ul><li>"Quels sont les avantages de l'apprentissage machine dans le secteur de la santé?"</li><li>'Comment puis-je optimiser les performances de mon site web?'</li><li>'What are the main challenges in cybersecurity?'</li></ul> |
68
- | lexical | <ul><li>'Quel est le numéro de téléphone du service client ou du customer support?'</li><li>'Comment fonctionne la blockchain?'</li><li>'How can I reset my user password?'</li></ul> |
69
 
70
  ## Evaluation
71
 
@@ -92,7 +96,7 @@ from setfit import SetFitModel
92
  # Download from the 🤗 Hub
93
  model = SetFitModel.from_pretrained("yaniseuranova/setfit-paraphrase-mpnet-base-v2-sst2")
94
  # Run inference
95
- preds = model("Who is the founder of Tesla Motors?")
96
  ```
97
 
98
  <!--
@@ -122,16 +126,14 @@ preds = model("Who is the founder of Tesla Motors?")
122
  ## Training Details
123
 
124
  ### Training Set Metrics
125
- | Training set | Min | Median | Max |
126
- |:-------------|:----|:-------|:----|
127
- | Word count | 4 | 8.7667 | 15 |
128
 
129
- | Label | Training Sample Count |
130
- |:--------------|:----------------------|
131
- | very_semantic | 39 |
132
- | semantic | 30 |
133
- | lexical | 26 |
134
- | very_lexical | 25 |
135
 
136
  ### Training Hyperparameters
137
  - batch_size: (16, 16)
@@ -151,66 +153,18 @@ preds = model("Who is the founder of Tesla Motors?")
151
  - load_best_model_at_end: True
152
 
153
  ### Training Results
154
- | Epoch | Step | Training Loss | Validation Loss |
155
- |:-------:|:--------:|:-------------:|:---------------:|
156
- | 0.0015 | 1 | 0.3698 | - |
157
- | 0.0749 | 50 | 0.2642 | - |
158
- | 0.1497 | 100 | 0.2307 | - |
159
- | 0.2246 | 150 | 0.1452 | - |
160
- | 0.2994 | 200 | 0.0772 | - |
161
- | 0.3743 | 250 | 0.0149 | - |
162
- | 0.4491 | 300 | 0.0036 | - |
163
- | 0.5240 | 350 | 0.0009 | - |
164
- | 0.5988 | 400 | 0.0009 | - |
165
- | 0.6737 | 450 | 0.0008 | - |
166
- | 0.7485 | 500 | 0.0006 | - |
167
- | 0.8234 | 550 | 0.0003 | - |
168
- | 0.8982 | 600 | 0.0003 | - |
169
- | 0.9731 | 650 | 0.0003 | - |
170
- | 1.0 | 668 | - | 0.0001 |
171
- | 1.0479 | 700 | 0.0002 | - |
172
- | 1.1228 | 750 | 0.0002 | - |
173
- | 1.1976 | 800 | 0.0002 | - |
174
- | 1.2725 | 850 | 0.0003 | - |
175
- | 1.3473 | 900 | 0.0003 | - |
176
- | 1.4222 | 950 | 0.0001 | - |
177
- | 1.4970 | 1000 | 0.0002 | - |
178
- | 1.5719 | 1050 | 0.0002 | - |
179
- | 1.6467 | 1100 | 0.0003 | - |
180
- | 1.7216 | 1150 | 0.0001 | - |
181
- | 1.7964 | 1200 | 0.0001 | - |
182
- | 1.8713 | 1250 | 0.0002 | - |
183
- | 1.9461 | 1300 | 0.0001 | - |
184
- | 2.0 | 1336 | - | 0.0001 |
185
- | 2.0210 | 1350 | 0.0001 | - |
186
- | 2.0958 | 1400 | 0.0001 | - |
187
- | 2.1707 | 1450 | 0.0002 | - |
188
- | 2.2455 | 1500 | 0.0002 | - |
189
- | 2.3204 | 1550 | 0.0001 | - |
190
- | 2.3952 | 1600 | 0.0001 | - |
191
- | 2.4701 | 1650 | 0.0002 | - |
192
- | 2.5449 | 1700 | 0.0001 | - |
193
- | 2.6198 | 1750 | 0.0001 | - |
194
- | 2.6946 | 1800 | 0.0001 | - |
195
- | 2.7695 | 1850 | 0.0001 | - |
196
- | 2.8443 | 1900 | 0.0001 | - |
197
- | 2.9192 | 1950 | 0.0001 | - |
198
- | 2.9940 | 2000 | 0.0001 | - |
199
- | 3.0 | 2004 | - | 0.0 |
200
- | 3.0689 | 2050 | 0.0001 | - |
201
- | 3.1437 | 2100 | 0.0001 | - |
202
- | 3.2186 | 2150 | 0.0001 | - |
203
- | 3.2934 | 2200 | 0.0001 | - |
204
- | 3.3683 | 2250 | 0.0001 | - |
205
- | 3.4431 | 2300 | 0.0001 | - |
206
- | 3.5180 | 2350 | 0.0001 | - |
207
- | 3.5928 | 2400 | 0.0001 | - |
208
- | 3.6677 | 2450 | 0.0001 | - |
209
- | 3.7425 | 2500 | 0.0001 | - |
210
- | 3.8174 | 2550 | 0.0001 | - |
211
- | 3.8922 | 2600 | 0.0001 | - |
212
- | 3.9671 | 2650 | 0.0001 | - |
213
- | **4.0** | **2672** | **-** | **0.0** |
214
 
215
  * The bold row denotes the saved checkpoint.
216
  ### Framework Versions
 
9
  metrics:
10
  - accuracy
11
  widget:
12
+ - text: What is the primary difference between homomorphic encryption and multi-party
13
+ computation in the context of secure multi-party computation protocols?
14
+ - text: How do organizations balance the need for innovation with the potential risks
15
+ and unintended consequences of emerging technologies?
16
+ - text: How doCompaniesbalanceIndividualCreativitywithTeamCollaboration to driveInnovationinthe
17
+ WORKPlace?
18
+ - text: How do companies balance the need for innovation with the risk of disrupting
19
+ their existing business models?
20
+ - text: What is the primary application of Natural Language Processing (NLP) in Google's
21
+ BERT language model, and how does it utilize masked language modeling to improve
22
+ contextual understanding?
23
  pipeline_tag: text-classification
24
  inference: true
25
  model-index:
 
54
  - **Sentence Transformer body:** [sentence-transformers/all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2)
55
  - **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
56
  - **Maximum Sequence Length:** 384 tokens
57
+ - **Number of Classes:** 2 classes
58
  <!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
59
  <!-- - **Language:** Unknown -->
60
  <!-- - **License:** Unknown -->
 
66
  - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
67
 
68
  ### Model Labels
69
+ | Label | Examples |
70
+ |:---------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
71
+ | semantic | <ul><li>'How do artificial intelligence systems navigate the trade-off between simplicity and accuracy when modeling complex real-world phenomena?'</li><li>'How do complex systems, consisting of many interconnected components, give rise to emergent properties that cannot be predicted from the characteristics of their individual parts?'</li><li>'How do complex systems, such as those found in nature and human societies, exhibit emergent properties that arise from the interactions of individual components?'</li></ul> |
72
+ | lexical | <ul><li>'What is the primary difference between a generative adversarial network (GAN) and a variational autoencoder (VAE) in deep learning?'</li><li>'What is the primary difference between a Decision Tree and a Random Forest in Machine Learning, and how do they alleviate overfitting?'</li><li>'What is the primary difference between a Bayesian neural network and a traditional feedforward neural network in the context of machine learning?'</li></ul> |
 
 
73
 
74
  ## Evaluation
75
 
 
96
  # Download from the 🤗 Hub
97
  model = SetFitModel.from_pretrained("yaniseuranova/setfit-paraphrase-mpnet-base-v2-sst2")
98
  # Run inference
99
+ preds = model("How doCompaniesbalanceIndividualCreativitywithTeamCollaboration to driveInnovationinthe WORKPlace?")
100
  ```
101
 
102
  <!--
 
126
  ## Training Details
127
 
128
  ### Training Set Metrics
129
+ | Training set | Min | Median | Max |
130
+ |:-------------|:----|:--------|:----|
131
+ | Word count | 5 | 18.8511 | 32 |
132
 
133
+ | Label | Training Sample Count |
134
+ |:---------|:----------------------|
135
+ | lexical | 23 |
136
+ | semantic | 24 |
 
 
137
 
138
  ### Training Hyperparameters
139
  - batch_size: (16, 16)
 
153
  - load_best_model_at_end: True
154
 
155
  ### Training Results
156
+ | Epoch | Step | Training Loss | Validation Loss |
157
+ |:-------:|:-------:|:-------------:|:---------------:|
158
+ | 0.0139 | 1 | 0.2662 | - |
159
+ | 0.6944 | 50 | 0.0007 | - |
160
+ | 1.0 | 72 | - | 0.0003 |
161
+ | 1.3889 | 100 | 0.0004 | - |
162
+ | 2.0 | 144 | - | 0.0001 |
163
+ | 2.0833 | 150 | 0.0003 | - |
164
+ | 2.7778 | 200 | 0.0002 | - |
165
+ | 3.0 | 216 | - | 0.0001 |
166
+ | 3.4722 | 250 | 0.0002 | - |
167
+ | **4.0** | **288** | **-** | **0.0001** |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
168
 
169
  * The bold row denotes the saved checkpoint.
170
  ### Framework Versions
config.json CHANGED
@@ -1,5 +1,5 @@
1
  {
2
- "_name_or_path": "checkpoints/step_2672",
3
  "architectures": [
4
  "MPNetModel"
5
  ],
 
1
  {
2
+ "_name_or_path": "checkpoints/step_288",
3
  "architectures": [
4
  "MPNetModel"
5
  ],
config_setfit.json CHANGED
@@ -1,9 +1,7 @@
1
  {
2
  "normalize_embeddings": false,
3
  "labels": [
4
- "very_semantic",
5
- "semantic",
6
  "lexical",
7
- "very_lexical"
8
  ]
9
  }
 
1
  {
2
  "normalize_embeddings": false,
3
  "labels": [
 
 
4
  "lexical",
5
+ "semantic"
6
  ]
7
  }
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e383b67c5d576d05d2fcff25a7c9988abf5d98876540eebb026b38637f51f3bc
3
  size 437967672
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8dedbddc75ebb08be5ba7197043ab354aa2000a2466f382af22d7a93a7995589
3
  size 437967672
model_head.pkl CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:d3224952c00d1183b6f7ededc25e464c290820fedd782232512f62f766ebb24b
3
- size 25655
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e9620192f6f9c9c643e965c1aa1dec6d39196685b3d03c44e788dd075ab17785
3
+ size 7039