FlukeTJ/distilbert-base-thai-cased-finetuned-sentiment

Model description

This model is a fine-tuned version of distilbert/distilbert-base-multilingual-cased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.5419
• Accuracy: 0.7606
• F1: 0.7599

Usage

Here's how you can use this model:

from transformers import AutoTokenizer, AutoModelForSequenceClassification

# Load model and tokenizer
model_name = "FlukeTJ/distilbert-base-thai-cased-finetuned-sentiment"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

# Function to predict sentiment
def predict_sentiment(text):
    inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True)
    outputs = model(**inputs)
    prediction_id = outputs.logits.argmax(-1).item()
    return model.config.id2label[prediction_id]

# Example usage
text = "ฉันชอบหนังเรื่องนี้มาก"  # "I like this movie very much"
sentiment = predict_sentiment(text)
print(f"Text: {text}")
print(f"Sentiment: {sentiment}")

Training and evaluation data

• Training data : 55385
• Evaluate data : 13847
• Test data : 17308

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 64
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 3

Training results

Step	Epoch	Training Loss	Validation Loss	Accuracy	F1
500	0.5774	0.7735	0.6476	0.6983	0.6967
1000	1.1547	0.6111	0.5911	0.7299	0.7280
1500	1.7321	0.564	0.5567	0.7549	0.7541
2000	2.3095	0.5138	0.5462	0.7610	0.7602
2500	2.8868	0.483	0.5414	0.7662	0.7651

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1