Hugging Face's logo

yaniseuranova
/
setfit-rag-hybrid-search-query-router

Text Classification
setfit
Safetensors
sentence-transformers
xlm-roberta
generated_from_setfit_trainer
Eval Results
Model card Files Community
yaniseuranova commited on
Commit
9eac52b
•
1 Parent(s): 663d131

Add SetFit model

Browse files
Files changed (5) hide show
  1. README.md +31 -29
  2. config.json +1 -1
  3. config_setfit.json +2 -2
  4. model.safetensors +1 -1
  5. model_head.pkl +2 -2
README.md CHANGED
@@ -48,7 +48,7 @@ The model has been trained using an efficient few-shot learning technique that i
48
  - **Sentence Transformer body:** [sentence-transformers/paraphrase-mpnet-base-v2](https://huggingface.co/sentence-transformers/paraphrase-mpnet-base-v2)
49
  - **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
50
  - **Maximum Sequence Length:** 512 tokens
51
- - **Number of Classes:** 4 classes
52
  <!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
53
  <!-- - **Language:** Unknown -->
54
  <!-- - **License:** Unknown -->
@@ -60,12 +60,14 @@ The model has been trained using an efficient few-shot learning technique that i
60
  - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
61
 
62
  ### Model Labels
63
- | Label | Examples |
64
- |:----------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
65
- | very_semantic_queries | <ul><li>'Quels sont les principes fondamentaux du développement durable?'</li><li>"Comment améliorer l'efficacité énergétique dans les bâtiments?"</li><li>'Combien de calories dans une pomme?'</li></ul> |
66
- | very_lexical | <ul><li>"Quelle est la capitale de l'Italie?"</li><li>"Qui est l'auteur de '1984'?"</li><li>'What is the current unemployment rate in France?'</li></ul> |
67
- | semantic_queries | <ul><li>"Quels sont les avantages de l'apprentissage machine dans le secteur de la santé?"</li><li>'Comment puis-je optimiser les performances de mon site web?'</li><li>'Comment fonctionne la blockchain?'</li></ul> |
68
- | lexical | <ul><li>'Quel est le numéro de téléphone du service client?'</li><li>'How can I reset my password?'</li><li>'What is the zip code for New York?'</li></ul> |
 
 
69
 
70
  ## Evaluation
71
 
@@ -124,14 +126,14 @@ preds = model("Comment rédiger un bon CV?")
124
  ### Training Set Metrics
125
  | Training set | Min | Median | Max |
126
  |:-------------|:----|:-------|:----|
127
- | Word count | 4 | 7.0667 | 13 |
128
 
129
  | Label | Training Sample Count |
130
  |:----------------------|:----------------------|
131
- | very_semantic_queries | 17 |
132
  | semantic_queries | 18 |
133
- | lexical_queries | 0 |
134
- | very_lexical | 16 |
135
 
136
  ### Training Hyperparameters
137
  - batch_size: (16, 16)
@@ -153,24 +155,24 @@ preds = model("Comment rédiger un bon CV?")
153
  ### Training Results
154
  | Epoch | Step | Training Loss | Validation Loss |
155
  |:-------:|:-------:|:-------------:|:---------------:|
156
- | 0.0060 | 1 | 0.4001 | - |
157
- | 0.3012 | 50 | 0.1902 | - |
158
- | 0.6024 | 100 | 0.0223 | - |
159
- | 0.9036 | 150 | 0.0008 | - |
160
- | 1.0 | 166 | - | 0.0009 |
161
- | 1.2048 | 200 | 0.001 | - |
162
- | 1.5060 | 250 | 0.0007 | - |
163
- | 1.8072 | 300 | 0.0006 | - |
164
- | 2.0 | 332 | - | 0.0003 |
165
- | 2.1084 | 350 | 0.0006 | - |
166
- | 2.4096 | 400 | 0.0003 | - |
167
- | 2.7108 | 450 | 0.0004 | - |
168
- | 3.0 | 498 | - | 0.0002 |
169
- | 3.0120 | 500 | 0.0002 | - |
170
- | 3.3133 | 550 | 0.0003 | - |
171
- | 3.6145 | 600 | 0.0003 | - |
172
- | 3.9157 | 650 | 0.0003 | - |
173
- | **4.0** | **664** | **-** | **0.0001** |
174
 
175
  * The bold row denotes the saved checkpoint.
176
  ### Framework Versions
 
48
  - **Sentence Transformer body:** [sentence-transformers/paraphrase-mpnet-base-v2](https://huggingface.co/sentence-transformers/paraphrase-mpnet-base-v2)
49
  - **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
50
  - **Maximum Sequence Length:** 512 tokens
51
+ - **Number of Classes:** 6 classes
52
  <!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
53
  <!-- - **Language:** Unknown -->
54
  <!-- - **License:** Unknown -->
 
60
  - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
61
 
62
  ### Model Labels
63
+ | Label | Examples |
64
+ |:----------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
65
+ | very_semantic_queries | <ul><li>'Quels sont les principes fondamentaux du corps humain?'</li><li>"Comment améliorer l'efficacité énergétique dans les bâtiments?"</li><li>'Combien de calories dans une pomme?'</li></ul> |
66
+ | very_lexical | <ul><li>"Quelle est la capitale de l'Italie?"</li><li>"Qui est l'auteur de '1984'?"</li><li>'What is the current unemployment rate in France?'</li></ul> |
67
+ | semantic_queries | <ul><li>"Quels sont les avantages de l'apprentissage machine dans le secteur de la santé?"</li><li>'Comment puis-je optimiser les performances de mon site web?'</li><li>'What are the main challenges in cybersecurity?'</li></ul> |
68
+ | lexical | <ul><li>'Quel est le numéro de téléphone du service client ou du customer suport?'</li><li>'How can I reset my user password?'</li><li>'What is the zip code for New York?'</li></ul> |
69
+ | lexical_queries | <ul><li>'Comment fonctionne la blockchain?'</li></ul> |
70
+ | lexical_query | <ul><li>'Who won the Nobel Peace Prize in 2021?'</li></ul> |
71
 
72
  ## Evaluation
73
 
 
126
  ### Training Set Metrics
127
  | Training set | Min | Median | Max |
128
  |:-------------|:----|:-------|:----|
129
+ | Word count | 4 | 7.1667 | 13 |
130
 
131
  | Label | Training Sample Count |
132
  |:----------------------|:----------------------|
133
+ | very_semantic_queries | 16 |
134
  | semantic_queries | 18 |
135
+ | lexical_queries | 1 |
136
+ | very_lexical | 15 |
137
 
138
  ### Training Hyperparameters
139
  - batch_size: (16, 16)
 
155
  ### Training Results
156
  | Epoch | Step | Training Loss | Validation Loss |
157
  |:-------:|:-------:|:-------------:|:---------------:|
158
+ | 0.0059 | 1 | 0.4006 | - |
159
+ | 0.2941 | 50 | 0.1896 | - |
160
+ | 0.5882 | 100 | 0.052 | - |
161
+ | 0.8824 | 150 | 0.0042 | - |
162
+ | 1.0 | 170 | - | 0.0023 |
163
+ | 1.1765 | 200 | 0.0011 | - |
164
+ | 1.4706 | 250 | 0.0006 | - |
165
+ | 1.7647 | 300 | 0.0007 | - |
166
+ | 2.0 | 340 | - | 0.0003 |
167
+ | 2.0588 | 350 | 0.0004 | - |
168
+ | 2.3529 | 400 | 0.0004 | - |
169
+ | 2.6471 | 450 | 0.0004 | - |
170
+ | 2.9412 | 500 | 0.0009 | - |
171
+ | 3.0 | 510 | - | 0.0003 |
172
+ | 3.2353 | 550 | 0.0003 | - |
173
+ | 3.5294 | 600 | 0.0004 | - |
174
+ | 3.8235 | 650 | 0.0003 | - |
175
+ | **4.0** | **680** | **-** | **0.0002** |
176
 
177
  * The bold row denotes the saved checkpoint.
178
  ### Framework Versions
config.json CHANGED
@@ -1,5 +1,5 @@
1
  {
2
- "_name_or_path": "checkpoints/step_664",
3
  "architectures": [
4
  "MPNetModel"
5
  ],
 
1
  {
2
+ "_name_or_path": "checkpoints/step_680",
3
  "architectures": [
4
  "MPNetModel"
5
  ],
config_setfit.json CHANGED
@@ -1,9 +1,9 @@
1
  {
2
- "normalize_embeddings": false,
3
  "labels": [
4
  "very_semantic_queries",
5
  "semantic_queries",
6
  "lexical_queries",
7
  "very_lexical"
8
- ]
 
9
  }
 
1
  {
 
2
  "labels": [
3
  "very_semantic_queries",
4
  "semantic_queries",
5
  "lexical_queries",
6
  "very_lexical"
7
+ ],
8
+ "normalize_embeddings": false
9
  }
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:d73c5537d6b9f04cb36ac62e63b4699fc99c23ee5198ef82ea0b2a5e052c607d
3
  size 437967672
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:90b2b4e9ab2e110c6b28701f1982fe75210d765ad667f050c5225fb0e218d56e
3
  size 437967672
model_head.pkl CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:04a12a12e3c2b789e754f1a0857c3b2603a951df4275fd12361e1c43036fc823
3
- size 25783
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f1b648aa4a967e622bed4524cadcef504ec05d246c86c8783556500585c58c06
3
+ size 38263