Hugging Face's logo

yaniseuranova
/
setfit-rag-hybrid-search-query-router-test

Text Classification
setfit
Safetensors
sentence-transformers
bert
generated_from_setfit_trainer
Eval Results
Model card Files Community
yaniseuranova commited on
Commit
1ce14f1
•
1 Parent(s): 9c3c65e

Add SetFit model

Browse files
Files changed (4) hide show
  1. README.md +95 -65
  2. config.json +1 -1
  3. model.safetensors +1 -1
  4. model_head.pkl +1 -1
README.md CHANGED
@@ -10,15 +10,14 @@ tags:
10
  - text-classification
11
  - generated_from_setfit_trainer
12
  widget:
13
- - text: What are the key components involved in developing a deep learning model for
14
- handwritten digit recognition?
 
 
 
15
  - text: What is the purpose of the message posted by the CR?
16
- - text: How can researchers create and maintain public repositories for reproducible
17
- research?
18
- - text: What are the key components involved in developing a deep learning model for
19
- handwritten digit recognition?
20
- - text: How do you prioritize and delegate tasks to ensure efficient collaboration
21
- and feedback?
22
  inference: true
23
  model-index:
24
  - name: SetFit with sentence-transformers/all-MiniLM-L6-v2
@@ -32,7 +31,7 @@ model-index:
32
  split: test
33
  metrics:
34
  - type: accuracy
35
- value: 0.5
36
  name: Accuracy
37
  ---
38
 
@@ -64,19 +63,19 @@ The model has been trained using an efficient few-shot learning technique that i
64
  - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
65
 
66
  ### Model Labels
67
- | Label | Examples |
68
- |:--------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
69
- | lexical | <ul><li>'What are the key considerations when choosing an optimization method for a complex problem?'</li><li>'What are the challenges of being a remote mentor or sponsor?'</li><li>'How do researchers typically obtain information on the ranking of machine learning conferences?'</li></ul> |
70
- | semantic | <ul><li>'What are common issues that users may encounter when accessing a platform that uses JumpCloud for authentication?'</li><li>'What are the key components involved in developing a deep learning model for handwritten digit recognition?'</li><li>'How can machine learning and data enrichment be used to improve business outcomes in various industries?'</li></ul> |
71
- | very_semantic | <ul><li>"What are people's opinions on a particular topic?"</li><li>'What are the key considerations when proposing names for a project or initiative?'</li><li>'What are the key considerations for successful collaboration between industry and academia in research and development projects?'</li></ul> |
72
- | very_lexical | <ul><li>'How can one track and store keys in a Flink operator?'</li><li>'What role do companies like Solvay play in addressing key societal challenges through their business strategies and operations?'</li><li>'What is the purpose of the scoring methodology in determining RAI maturity?'</li></ul> |
73
 
74
  ## Evaluation
75
 
76
  ### Metrics
77
  | Label | Accuracy |
78
  |:--------|:---------|
79
- | **all** | 0.5 |
80
 
81
  ## Uses
82
 
@@ -128,14 +127,14 @@ preds = model("What is the purpose of the message posted by the CR?")
128
  ### Training Set Metrics
129
  | Training set | Min | Median | Max |
130
  |:-------------|:----|:--------|:----|
131
- | Word count | 8 | 14.4138 | 24 |
132
 
133
  | Label | Training Sample Count |
134
  |:--------------|:----------------------|
135
- | lexical | 32 |
136
- | semantic | 21 |
137
- | very_lexical | 10 |
138
- | very_semantic | 24 |
139
 
140
  ### Training Hyperparameters
141
  - batch_size: (8, 8)
@@ -157,50 +156,81 @@ preds = model("What is the purpose of the message posted by the CR?")
157
  ### Training Results
158
  | Epoch | Step | Training Loss | Validation Loss |
159
  |:-------:|:--------:|:-------------:|:---------------:|
160
- | 0.0015 | 1 | 0.268 | - |
161
- | 0.0736 | 50 | 0.2649 | - |
162
- | 0.1473 | 100 | 0.3352 | - |
163
- | 0.2209 | 150 | 0.2516 | - |
164
- | 0.2946 | 200 | 0.2438 | - |
165
- | 0.3682 | 250 | 0.1808 | - |
166
- | 0.4418 | 300 | 0.2365 | - |
167
- | 0.5155 | 350 | 0.1337 | - |
168
- | 0.5891 | 400 | 0.2263 | - |
169
- | 0.6627 | 450 | 0.1936 | - |
170
- | 0.7364 | 500 | 0.0612 | - |
171
- | 0.8100 | 550 | 0.1664 | - |
172
- | 0.8837 | 600 | 0.0987 | - |
173
- | 0.9573 | 650 | 0.0736 | - |
174
- | 1.0 | 679 | - | 0.2288 |
175
- | 1.0309 | 700 | 0.0568 | - |
176
- | 1.1046 | 750 | 0.0765 | - |
177
- | 1.1782 | 800 | 0.1193 | - |
178
- | 1.2518 | 850 | 0.199 | - |
179
- | 1.3255 | 900 | 0.2734 | - |
180
- | 1.3991 | 950 | 0.194 | - |
181
- | 1.4728 | 1000 | 0.1085 | - |
182
- | 1.5464 | 1050 | 0.1496 | - |
183
- | 1.6200 | 1100 | 0.1673 | - |
184
- | 1.6937 | 1150 | 0.2225 | - |
185
- | 1.7673 | 1200 | 0.0503 | - |
186
- | 1.8409 | 1250 | 0.1531 | - |
187
- | 1.9146 | 1300 | 0.2287 | - |
188
- | 1.9882 | 1350 | 0.1187 | - |
189
- | **2.0** | **1358** | **-** | **0.2055** |
190
- | 2.0619 | 1400 | 0.0546 | - |
191
- | 2.1355 | 1450 | 0.2072 | - |
192
- | 2.2091 | 1500 | 0.1208 | - |
193
- | 2.2828 | 1550 | 0.0837 | - |
194
- | 2.3564 | 1600 | 0.0405 | - |
195
- | 2.4300 | 1650 | 0.1334 | - |
196
- | 2.5037 | 1700 | 0.1458 | - |
197
- | 2.5773 | 1750 | 0.2189 | - |
198
- | 2.6510 | 1800 | 0.0561 | - |
199
- | 2.7246 | 1850 | 0.1656 | - |
200
- | 2.7982 | 1900 | 0.1351 | - |
201
- | 2.8719 | 1950 | 0.1826 | - |
202
- | 2.9455 | 2000 | 0.1905 | - |
203
- | 3.0 | 2037 | - | 0.2273 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
204
 
205
  * The bold row denotes the saved checkpoint.
206
  ### Framework Versions
 
10
  - text-classification
11
  - generated_from_setfit_trainer
12
  widget:
13
+ - text: What are the key situations that require the preparation of a mission order?
14
+ - text: How can audio data be used to improve speaker identification using neural
15
+ networks?
16
+ - text: How can organizations balance the need for data privacy with the benefits
17
+ of involving interns in data-related projects?
18
  - text: What is the purpose of the message posted by the CR?
19
+ - text: What are the consequences of adopting a 'if not broken, don't fix' attitude
20
+ towards data monitoring?
 
 
 
 
21
  inference: true
22
  model-index:
23
  - name: SetFit with sentence-transformers/all-MiniLM-L6-v2
 
31
  split: test
32
  metrics:
33
  - type: accuracy
34
+ value: 0.3076923076923077
35
  name: Accuracy
36
  ---
37
 
 
63
  - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
64
 
65
  ### Model Labels
66
+ | Label | Examples |
67
+ |:--------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
68
+ | very_semantic | <ul><li>'What are the key considerations when proposing names for a project or initiative?'</li><li>'What are the key aspects of team life and events in a company?'</li><li>'What is being asked for or sought in this conversation?'</li></ul> |
69
+ | lexical | <ul><li>'Who is responsible for reviewing and signing documents related to conference submissions?'</li><li>'How do data architecture and management systems enable digital transformation and address its associated challenges?'</li><li>'How do keys or access credentials get shared or transferred among team members in a workplace?'</li></ul> |
70
+ | very_lexical | <ul><li>'What are some of the key challenges associated with handling and storing large amounts of genomic data?'</li><li>"What is the focus of Eurobiomed's partnership with Digital113?"</li><li>'What are the key considerations for generating well-formatted JSON instances that conform to a given schema?'</li></ul> |
71
+ | semantic | <ul><li>'How can visualizations be used to enhance documentation and collaboration in software development?'</li><li>'What are the key considerations when choosing a distance metric for a vector database?'</li><li>'How can AI be leveraged to support HR departments in detecting and addressing gender bias?'</li></ul> |
72
 
73
  ## Evaluation
74
 
75
  ### Metrics
76
  | Label | Accuracy |
77
  |:--------|:---------|
78
+ | **all** | 0.3077 |
79
 
80
  ## Uses
81
 
 
127
  ### Training Set Metrics
128
  | Training set | Min | Median | Max |
129
  |:-------------|:----|:--------|:----|
130
+ | Word count | 7 | 14.1913 | 24 |
131
 
132
  | Label | Training Sample Count |
133
  |:--------------|:----------------------|
134
+ | lexical | 41 |
135
+ | semantic | 24 |
136
+ | very_lexical | 17 |
137
+ | very_semantic | 33 |
138
 
139
  ### Training Hyperparameters
140
  - batch_size: (8, 8)
 
156
  ### Training Results
157
  | Epoch | Step | Training Loss | Validation Loss |
158
  |:-------:|:--------:|:-------------:|:---------------:|
159
+ | 0.0008 | 1 | 0.4237 | - |
160
+ | 0.0417 | 50 | 0.2917 | - |
161
+ | 0.0834 | 100 | 0.1835 | - |
162
+ | 0.1251 | 150 | 0.3215 | - |
163
+ | 0.1668 | 200 | 0.2299 | - |
164
+ | 0.2085 | 250 | 0.2595 | - |
165
+ | 0.2502 | 300 | 0.3193 | - |
166
+ | 0.2919 | 350 | 0.2288 | - |
167
+ | 0.3336 | 400 | 0.2947 | - |
168
+ | 0.3753 | 450 | 0.1171 | - |
169
+ | 0.4170 | 500 | 0.1442 | - |
170
+ | 0.4587 | 550 | 0.1859 | - |
171
+ | 0.5004 | 600 | 0.1959 | - |
172
+ | 0.5421 | 650 | 0.2797 | - |
173
+ | 0.5838 | 700 | 0.2079 | - |
174
+ | 0.6255 | 750 | 0.2706 | - |
175
+ | 0.6672 | 800 | 0.1956 | - |
176
+ | 0.7089 | 850 | 0.0833 | - |
177
+ | 0.7506 | 900 | 0.1421 | - |
178
+ | 0.7923 | 950 | 0.2345 | - |
179
+ | 0.8340 | 1000 | 0.1347 | - |
180
+ | 0.8757 | 1050 | 0.241 | - |
181
+ | 0.9174 | 1100 | 0.133 | - |
182
+ | 0.9591 | 1150 | 0.1041 | - |
183
+ | **1.0** | **1199** | **-** | **0.3562** |
184
+ | 1.0008 | 1200 | 0.0837 | - |
185
+ | 1.0425 | 1250 | 0.1566 | - |
186
+ | 1.0842 | 1300 | 0.2101 | - |
187
+ | 1.1259 | 1350 | 0.0496 | - |
188
+ | 1.1676 | 1400 | 0.063 | - |
189
+ | 1.2093 | 1450 | 0.149 | - |
190
+ | 1.2510 | 1500 | 0.038 | - |
191
+ | 1.2927 | 1550 | 0.0504 | - |
192
+ | 1.3344 | 1600 | 0.0679 | - |
193
+ | 1.3761 | 1650 | 0.1699 | - |
194
+ | 1.4178 | 1700 | 0.1293 | - |
195
+ | 1.4595 | 1750 | 0.1083 | - |
196
+ | 1.5013 | 1800 | 0.2044 | - |
197
+ | 1.5430 | 1850 | 0.1267 | - |
198
+ | 1.5847 | 1900 | 0.0842 | - |
199
+ | 1.6264 | 1950 | 0.1126 | - |
200
+ | 1.6681 | 2000 | 0.0544 | - |
201
+ | 1.7098 | 2050 | 0.143 | - |
202
+ | 1.7515 | 2100 | 0.08 | - |
203
+ | 1.7932 | 2150 | 0.1103 | - |
204
+ | 1.8349 | 2200 | 0.1768 | - |
205
+ | 1.8766 | 2250 | 0.1639 | - |
206
+ | 1.9183 | 2300 | 0.1637 | - |
207
+ | 1.9600 | 2350 | 0.1637 | - |
208
+ | 2.0 | 2398 | - | 0.3682 |
209
+ | 2.0017 | 2400 | 0.2938 | - |
210
+ | 2.0434 | 2450 | 0.0808 | - |
211
+ | 2.0851 | 2500 | 0.0788 | - |
212
+ | 2.1268 | 2550 | 0.2187 | - |
213
+ | 2.1685 | 2600 | 0.0701 | - |
214
+ | 2.2102 | 2650 | 0.0385 | - |
215
+ | 2.2519 | 2700 | 0.135 | - |
216
+ | 2.2936 | 2750 | 0.2276 | - |
217
+ | 2.3353 | 2800 | 0.2203 | - |
218
+ | 2.3770 | 2850 | 0.0029 | - |
219
+ | 2.4187 | 2900 | 0.1855 | - |
220
+ | 2.4604 | 2950 | 0.1278 | - |
221
+ | 2.5021 | 3000 | 0.0487 | - |
222
+ | 2.5438 | 3050 | 0.0404 | - |
223
+ | 2.5855 | 3100 | 0.1158 | - |
224
+ | 2.6272 | 3150 | 0.1354 | - |
225
+ | 2.6689 | 3200 | 0.1633 | - |
226
+ | 2.7106 | 3250 | 0.1484 | - |
227
+ | 2.7523 | 3300 | 0.1146 | - |
228
+ | 2.7940 | 3350 | 0.1437 | - |
229
+ | 2.8357 | 3400 | 0.0948 | - |
230
+ | 2.8774 | 3450 | 0.0833 | - |
231
+ | 2.9191 | 3500 | 0.0668 | - |
232
+ | 2.9608 | 3550 | 0.1687 | - |
233
+ | 3.0 | 3597 | - | 0.3651 |
234
 
235
  * The bold row denotes the saved checkpoint.
236
  ### Framework Versions
config.json CHANGED
@@ -1,5 +1,5 @@
1
  {
2
- "_name_or_path": "checkpoints/step_1358",
3
  "architectures": [
4
  "BertModel"
5
  ],
 
1
  {
2
+ "_name_or_path": "checkpoints/step_1199",
3
  "architectures": [
4
  "BertModel"
5
  ],
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:7fac62744a83855a95a3e80c70bf8a4648a3c5a1cd0053760fa1ff330790c771
3
  size 90864192
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:54e1f2ecdb4d01b6727aa7c7082233ca8c6ed1ad2689e8555996d2feeeeb4e57
3
  size 90864192
model_head.pkl CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:a5a2800b0ffabd217138abf7b9e4a3321ce002b79f4c83251f28a4f0a7a58788
3
  size 13367
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:178c77f056f085ee38b4a6c61f668e1dedcdd7a98a79ec3d6055de6ef300abf3
3
  size 13367