ai-chat-censor6

The primary focus of the model is detecting sexual/minors category of messages.

Possible flags are: regular, racist, underage, sexual

BEWARE

The model might categorize any talk about race as racism, for example: "Black people suffer so much in America" will be flagged as "racist".

Model also might flag any comment containing numbers below 18 as underage. This is an issue that will be addressed in next version.

Here's the next version: https://huggingface.co./andriadze/ai-chat-censor

Training and evaluation data

Model was trained on a fully synthetic dataset generated by uncensored 72b models based on qwen2.

This model is a fine-tuned version of distilbert/distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0637
• Accuracy: 0.9903

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 6

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
0.0471	1.0	1175	0.0729	0.9854
0.0282	2.0	2350	0.0529	0.9900
0.0105	3.0	3525	0.0680	0.9888
0.0079	4.0	4700	0.0558	0.9911
0.0017	5.0	5875	0.0595	0.9902
0.0001	6.0	7050	0.0637	0.9903

Framework versions

• Transformers 4.44.2
• Pytorch 2.4.0+cu121
• Datasets 3.0.0
• Tokenizers 0.19.1