README.md

metadata

datasets:
  - go_emotions
language:
  - en
library_name: transformers
model-index:
  - name: text-classification-goemotions
    results:
      - task:
          name: Text Classification
          type: text-classification
        dataset:
          name: go_emotions
          type: multilabel_classification
          config: simplified
          split: test
          args: simplified
        metrics:
          - name: F1
            type: f1
            value: 0.487

Text Classification GoEmotions

This model is a fined-tuned version of nreimers/MiniLMv2-L6-H384-distilled-from-RoBERTa-Large on the on the go_emotions dataset using tasinho/text-classification-goemotions as teacher model. The quantized version in ONNX format can be found here

Load the Model

from transformers import pipeline

pipe = pipeline(model='minuva/MiniLMv2-goemotions-v2', task='text-classification')
pipe("I am angry")
# [{'label': 'anger', 'score': 0.9722517132759094}]

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 6e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear

Metrics (comparison with teacher model)

Teacher (params)	Student (params)	Set	Score (teacher)	Score (student)
tasinhoque/text-classification-goemotions (355M)	MiniLMv2-L6-H384-goemotions-v2	Validation	0.514252	0.484898
tasinhoque/text-classification-goemotions (33M)	MiniLMv2-L6-H384-goemotions-v2	Test	0.501937	0.486890

Deployment

Check our repository to see how to easily deploy this (quantized) model in a serverless environment with fast CPU inference and light resource utilization.