assets/README.md

metadata

library_name: setfit
tags:
  - setfit
  - sentence-transformers
  - text-classification
  - generated_from_setfit_trainer
base_model: BAAI/bge-base-en-v1.5
metrics:
  - accuracy
widget:
  - text: >-
      Dear GO FIRST Flyer, Comply with web-check in & state regulations on
      www.FlyGoFirst.com/wci. Power banks are allowed ONLY in cabin baggage and
      NOT in check-in baggage.
  - text: >-
      INR 415000.00 debited from DBS a/c no. ********1417 on 09-04-2022 to
      mishra ji via NEFT (UTR Ref No 0811OP2104023743) will reach bene a/c
      usually within 2 hours.
  - text: >-
      Your plan Rs 719-3m-2GB/D for Jio Number 8076716202 has expired on
      12-Feb-23 22:24 Hrs. To continue enjoying Jio services, recharge
      immediately.  To recharge, click https://www.jio.com/selfcare/recharge
      Dial 1991, to know your current balance, validity, plan details and for
      exciting recharge plans.
  - text: >-
      HDFC Bank: Rs 72.00 debited from a/c **0591 on 16-10-21 to VPA
      gpay-11169632313@okbizaxis(UPI Ref No 128968285337). Not you? Call on
      18002586161 to report
  - text: >-
      Get up to Rs.1K assured cashback & Rs.25K free credit with OlaMoney
      Postpaid+. Use it across 15K+ apps, pay back in 30 days. Click :
      https://hello.ola.app/ompps
pipeline_tag: text-classification
inference: true
model-index:
  - name: SetFit with BAAI/bge-base-en-v1.5
    results:
      - task:
          type: text-classification
          name: Text Classification
        dataset:
          name: Unknown
          type: unknown
          split: test
        metrics:
          - type: accuracy
            value: 0.9576271186440678
            name: Accuracy

SetFit with BAAI/bge-base-en-v1.5

This is a SetFit model that can be used for Text Classification. This SetFit model uses BAAI/bge-base-en-v1.5 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

1. Fine-tuning a Sentence Transformer with contrastive learning.
2. Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

• Model Type: SetFit
• Sentence Transformer body: BAAI/bge-base-en-v1.5
• Classification head: a LogisticRegression instance
• Maximum Sequence Length: 512 tokens
• Number of Classes: 2 classes

Model Sources

• Repository: SetFit on GitHub
• Paper: Efficient Few-Shot Learning Without Prompts
• Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
Offer	'Exclusive offer only for you.Recharge with Rs.49 to get 100min voice call,1GB data for 15 days.Hurry,recharge today.' 'Abhi *911# milaa kr payen Rs.25 advance loan aur phir 51# dial kren aur Rs. 10+T mei haasil kren, 500 MB aur 50 Zong Minutes puray din kay liye' 'HDFC BANK PERSONAL L0AN IndependenceDay offer 10.25% 0%PF FRESH LOAN / TOPUP LOAN / PARALLEL LOAN/BT Hurry! Apply - 8095227619 T&C '
Transaction	'Dear Customer, your DBS account no ***1417 is credited with INR 25000 on 01-10-2021 and is subject to clearance. Current Balance is INR 58661.69.' 'Pls use IFSC BARB0 instead of old IFSC VIJB for remittances as old code will be discontinued wef 01.07.2021. Advise your remitters also-Bank of Baroda' 'Your Stock broker ZERODHA BROKING LIMITED. reported your fund balance Rs.26986.54 & securities balance 0 as on end of 09-oct-2021. Balances do not cover your bank, DP & PMS balance with broking entity. Check details at [email protected]. If email Id not correct, kindly update with your broker - National Stock Exchange'

Evaluation

Metrics

Label	Accuracy
all	0.9576

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("setfit_model_id")
# Run inference
preds = model("HDFC Bank: Rs 72.00 debited from a/c **0591 on 16-10-21 to VPA gpay-11169632313@okbizaxis(UPI Ref No 128968285337). Not you? Call on 18002586161 to report")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	3	26.7702	135

Label	Training Sample Count
Transaction	103
Offer	132

Training Hyperparameters

• batch_size: (16, 16)
• num_epochs: (1, 1)
• max_steps: -1
• sampling_strategy: oversampling
• num_iterations: 1
• body_learning_rate: (2e-05, 1e-05)
• head_learning_rate: 0.01
• loss: CosineSimilarityLoss
• distance_metric: cosine_distance
• margin: 0.25
• end_to_end: False
• use_amp: False
• warmup_proportion: 0.1
• seed: 42
• eval_max_steps: -1
• load_best_model_at_end: False

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0333	1	0.2366	-

Framework Versions

• Python: 3.11.9
• SetFit: 1.0.3
• Sentence Transformers: 3.0.1
• Transformers: 4.39.0
• PyTorch: 2.3.1+cu121
• Datasets: 2.19.2
• Tokenizers: 0.15.2

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}