Hugging Face's logo

vipinbansal179
/
SetFit_sms_Analyzer5c95292

Text Classification
setfit
Safetensors
sentence-transformers
mpnet
generated_from_setfit_trainer
Eval Results
Model card Files Community
vipinbansal179 commited on
Commit
f70f6ac
•
1 Parent(s): 6b599de

Add SetFit model

Browse files
Files changed (4) hide show
  1. README.md +20 -45
  2. config.json +1 -1
  3. model.safetensors +1 -1
  4. model_head.pkl +1 -1
README.md CHANGED
@@ -32,7 +32,7 @@ model-index:
32
  split: test
33
  metrics:
34
  - type: accuracy
35
- value: 0.9722222222222222
36
  name: Accuracy
37
  ---
38
 
@@ -64,18 +64,18 @@ The model has been trained using an efficient few-shot learning technique that i
64
  - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
65
 
66
  ### Model Labels
67
- | Label | Examples |
68
- |:------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
69
- | 2 | <ul><li>'validity airtel xstream fiber id 20001896982 expire 04-sep-23 . please recharge rs 589 enjoy uninterrupted service . recharge , click www.airtel.in/5/c_summary ? n=021710937343_dsl . please ignore already pay .'</li><li>'initiate process add a/c . xxxx59 upi app - axis bank'</li><li>'google-pay registration initiate icici bank . do , report bank . card details/otp/cvv secret . disclose anyone .'</li></ul> |
70
- | 0 | <ul><li>'rs 260.00 debit a/c xxxxxx7783 credit krjngm @ oksbi upi ref:325154274303. ? call 18005700 -bob'</li><li>'send rs.400.00 kotak bank ac x4524 7800600122 @ ybl 15-10-23.upi ref 328855774953. , kotak.com/fraud'</li><li>'send rs.400.00 kotak bank ac x4524 7800600122 @ ybl 15-10-23.upi ref 328855774953. , kotak.com/fraud'</li></ul> |
71
- | 1 | <ul><li>'dear bob upi user , account credit inr 50.00 date 2023-08-27 11:41:09 upi ref 360562629741 - bob'</li><li>'receive rs.10000.00 kotak bank ac x4524 mahimagyamlani08 @ okaxis 21-08-23.bal:197,838.14.upi ref:323334598750'</li><li>'update ! inr5.66 credit federal bank account xxxx9374 jupiter app . happy bank !'</li></ul> |
72
 
73
  ## Evaluation
74
 
75
  ### Metrics
76
  | Label | Accuracy |
77
  |:--------|:---------|
78
- | **all** | 0.9722 |
79
 
80
  ## Uses
81
 
@@ -127,17 +127,17 @@ preds = model("< # > use otp : 8233 login turtlemintpro zck+rfoaqnm")
127
  ### Training Set Metrics
128
  | Training set | Min | Median | Max |
129
  |:-------------|:----|:--------|:----|
130
- | Word count | 5 | 20.5357 | 35 |
131
 
132
  | Label | Training Sample Count |
133
  |:------|:----------------------|
134
- | 0 | 31 |
135
- | 1 | 28 |
136
- | 2 | 81 |
137
 
138
  ### Training Hyperparameters
139
  - batch_size: (16, 16)
140
- - num_epochs: (2, 2)
141
  - max_steps: -1
142
  - sampling_strategy: oversampling
143
  - body_learning_rate: (2e-05, 1e-05)
@@ -153,39 +153,14 @@ preds = model("< # > use otp : 8233 login turtlemintpro zck+rfoaqnm")
153
  - load_best_model_at_end: True
154
 
155
  ### Training Results
156
- | Epoch | Step | Training Loss | Validation Loss |
157
- |:-------:|:-------:|:-------------:|:---------------:|
158
- | 0.0014 | 1 | 0.2939 | - |
159
- | 0.0708 | 50 | 0.1406 | - |
160
- | 0.1416 | 100 | 0.0304 | - |
161
- | 0.2125 | 150 | 0.0078 | - |
162
- | 0.2833 | 200 | 0.0019 | - |
163
- | 0.3541 | 250 | 0.0001 | - |
164
- | 0.4249 | 300 | 0.0003 | - |
165
- | 0.4958 | 350 | 0.0001 | - |
166
- | 0.5666 | 400 | 0.0001 | - |
167
- | 0.6374 | 450 | 0.0001 | - |
168
- | 0.7082 | 500 | 0.0001 | - |
169
- | 0.7790 | 550 | 0.0001 | - |
170
- | 0.8499 | 600 | 0.0002 | - |
171
- | 0.9207 | 650 | 0.0001 | - |
172
- | 0.9915 | 700 | 0.0001 | - |
173
- | **1.0** | **706** | **-** | **0.0168** |
174
- | 1.0623 | 750 | 0.0 | - |
175
- | 1.1331 | 800 | 0.0001 | - |
176
- | 1.2040 | 850 | 0.0001 | - |
177
- | 1.2748 | 900 | 0.0 | - |
178
- | 1.3456 | 950 | 0.0001 | - |
179
- | 1.4164 | 1000 | 0.0 | - |
180
- | 1.4873 | 1050 | 0.0 | - |
181
- | 1.5581 | 1100 | 0.0 | - |
182
- | 1.6289 | 1150 | 0.0 | - |
183
- | 1.6997 | 1200 | 0.0 | - |
184
- | 1.7705 | 1250 | 0.0 | - |
185
- | 1.8414 | 1300 | 0.0002 | - |
186
- | 1.9122 | 1350 | 0.0 | - |
187
- | 1.9830 | 1400 | 0.0 | - |
188
- | 2.0 | 1412 | - | 0.0183 |
189
 
190
  * The bold row denotes the saved checkpoint.
191
  ### Framework Versions
 
32
  split: test
33
  metrics:
34
  - type: accuracy
35
+ value: 0.6944444444444444
36
  name: Accuracy
37
  ---
38
 
 
64
  - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
65
 
66
  ### Model Labels
67
+ | Label | Examples |
68
+ |:------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
69
+ | 2 | <ul><li>'hi ashwin , credit score qualify personal loan rs.10 lacs . apply get 10 mins https : //tneu.in/qilih6y4 & c apply - tataneu'</li><li>'592104 otp transaction inr 589.00 kotak bank card 6343 airtel , valid 3 mins . share otp anyone .'</li><li>'dear customer , get axis vistara credit card , complimentary economy ticket join & upto 4 ticket yearly , apply : https : //1kx.in/xrb8ksi6tkf loanmandi'</li></ul> |
70
+ | 0 | <ul><li>"'ve spend rs.17220.00 thru kotak bank debit card xx2169 inf * www.vfsglobal.com 12/10/2023 avl bal 196168.62 ? visit kotak.com/fraud"</li><li>'rs 260.00 debit a/c xxxxxx7783 credit krjngm @ oksbi upi ref:325154274303. ? call 18005700 -bob'</li><li>'rs.381.56 transfer a/c ... 7783 : upi/327734499842 . total bal : rs.27615.37cr . avlbl amt : rs.27615.37 ( 04-10-2023 19:07:22 ) - bank baroda'</li></ul> |
71
+ | 1 | <ul><li>'update ! inr5.66 credit federal bank account xxxx9374 jupiter app . happy bank !'</li><li>'dear customer , inr 619.00 credit a/c xx7556 31/08/2023 neft utr cms3535305583 fintech blue solutions private limit , info : fintech blue solutions private limi ted-sbi'</li></ul> |
72
 
73
  ## Evaluation
74
 
75
  ### Metrics
76
  | Label | Accuracy |
77
  |:--------|:---------|
78
+ | **all** | 0.6944 |
79
 
80
  ## Uses
81
 
 
127
  ### Training Set Metrics
128
  | Training set | Min | Median | Max |
129
  |:-------------|:----|:--------|:----|
130
+ | Word count | 14 | 22.0417 | 30 |
131
 
132
  | Label | Training Sample Count |
133
  |:------|:----------------------|
134
+ | 0 | 6 |
135
+ | 1 | 2 |
136
+ | 2 | 16 |
137
 
138
  ### Training Hyperparameters
139
  - batch_size: (16, 16)
140
+ - num_epochs: (4, 4)
141
  - max_steps: -1
142
  - sampling_strategy: oversampling
143
  - body_learning_rate: (2e-05, 1e-05)
 
153
  - load_best_model_at_end: True
154
 
155
  ### Training Results
156
+ | Epoch | Step | Training Loss | Validation Loss |
157
+ |:-------:|:------:|:-------------:|:---------------:|
158
+ | 0.05 | 1 | 0.279 | - |
159
+ | **1.0** | **20** | **-** | **0.1602** |
160
+ | 2.0 | 40 | - | 0.1603 |
161
+ | 2.5 | 50 | 0.0013 | - |
162
+ | 3.0 | 60 | - | 0.1661 |
163
+ | 4.0 | 80 | - | 0.1706 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
164
 
165
  * The bold row denotes the saved checkpoint.
166
  ### Framework Versions
config.json CHANGED
@@ -1,5 +1,5 @@
1
  {
2
- "_name_or_path": "checkpoints/step_706/",
3
  "architectures": [
4
  "MPNetModel"
5
  ],
 
1
  {
2
+ "_name_or_path": "checkpoints/step_20/",
3
  "architectures": [
4
  "MPNetModel"
5
  ],
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:413446a1820462c7da2b156d339e1c8201a89373d3185937cf9e23c5453d757c
3
  size 437967672
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fc686d4ee9727ab20a11e61fec42072f6a2026a08427a68d92658f5c1ae2e37d
3
  size 437967672
model_head.pkl CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:16e50a8f183198bf58f8007ad172622255f6cb74bc1ef89d842157a740f93833
3
  size 19311
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:84d61b69473b85170ab65e6b7e07704d6a59f5e03637221cab5346fa22a834fe
3
  size 19311