File size: 5,441 Bytes
1cdaf3f 6b3ad6c 38bc26f 6b3ad6c 1cdaf3f df3c75d d6b0ebb df3c75d |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 |
---
language:
- en
metrics:
- accuracy
widget:
- text: "You have the right to use CommunityConnect for its intended purpose of connecting with others, sharing content responsibly, and engaging in constructive dialogue. You are responsible for the content you post and must respect the rights and privacy of others."
example_title: "Fair Clause"
- text: " We reserve the right to suspend, terminate, or restrict your access to the platform at any time and for any reason, without prior notice or explanation. This includes but is not limited to violations of our community guidelines or terms of service, as determined solely by ConnectWorld."
example_title: "Unfair Clause"
library_name: transformers
pipeline_tag: text-classification
tags:
- tos
- terms of services
- bert
---
# TOSBert
**TOSBert** is a fine-tuned BERT model for sequence classification tasks. It is trained on a custom dataset for multi-label classification.
## Model Details
- **Model Name**: TOSBert
- **Model Architecture**: BERT
- **Framework**: [Hugging Face Transformers](https://huggingface.co./transformers/)
- **Model Type**: Sequence Classification (Multi-label Classification)
## Dataset
The model is trained on the [online_terms_of_service](https://huggingface.co./datasets/joelniklaus/online_terms_of_service) dataset hosted on Hugging Face. This dataset consists of text sequences extracted from various online terms of service documents. Each sequence is labeled with multiple categories related to legal and privacy terms.
## Training
The model was fine-tuned using the following parameters:
- **Number of Epochs**: 3
- **Batch Size**: 16 (both for training and evaluation)
- **Warmup Steps**: 500
- **Weight Decay**: 0.01
- **Learning Rate**: Automatically adjusted
## Usage
### Installation
To use this model, you need to install the `transformers` library from Hugging Face:
```bash
pip install transformers
```
### Loading the Model
You can load the model using the following code:
```python
from transformers import BertForSequenceClassification, BertTokenizer
model_name = "CodeHima/TOSBert"
model = BertForSequenceClassification.from_pretrained(model_name)
tokenizer = BertTokenizer.from_pretrained(model_name)
```
### Inference
Here is an example of how to use the model for inference:
```python
from transformers import pipeline
classifier = pipeline("text-classification", model=model, tokenizer=tokenizer, return_all_scores=True)
text = "Your input text here"
predictions = classifier(text)
print(predictions)
```
### Training Script
Below is an example script used for training the model:
```python
from transformers import Trainer, TrainingArguments, BertForSequenceClassification, BertTokenizer
import torch
from sklearn.metrics import accuracy_score, precision_recall_fscore_support
# Define the model
model_name = "bert-base-uncased"
model = BertForSequenceClassification.from_pretrained(model_name, num_labels=3)
# Define the tokenizer
tokenizer = BertTokenizer.from_pretrained(model_name)
# Load your dataset
# train_dataset and eval_dataset should be instances of torch.utils.data.Dataset
# Example: train_dataset = YourDataset(train_data)
# Define training arguments
training_args = TrainingArguments(
output_dir='./results',
num_train_epochs=3,
per_device_train_batch_size=16,
per_device_eval_batch_size=16,
warmup_steps=500,
weight_decay=0.01,
logging_dir='./logs',
logging_steps=10,
eval_strategy="epoch"
)
# Custom data collator to convert labels to floats
def data_collator(features):
batch = {}
first = features[0]
if 'label' in first and first['label'] is not None:
dtype = torch.float32
batch['labels'] = torch.tensor([f['label'] for f in features], dtype=dtype)
for k, v in first.items():
if k != 'label' and v is not None and not isinstance(v, str):
batch[k] = torch.stack([f[k] for f in features])
return batch
# Define the compute metrics function
def compute_metrics(pred):
labels = pred.label_ids
preds = (pred.predictions > 0.5).astype(int)
precision, recall, f1, _ = precision_recall_fscore_support(labels, preds, average='micro')
acc = accuracy_score(labels, preds)
return {
'accuracy': acc,
'f1': f1,
'precision': precision,
'recall': recall
}
# Initialize the Trainer
trainer = Trainer(
model=model,
args=training_args,
train_dataset=train_dataset,
eval_dataset=eval_dataset,
compute_metrics=compute_metrics,
data_collator=data_collator
)
# Train the model
trainer.train()
```
## Evaluation
To evaluate the model on the validation set, you can use the following code:
```python
results = trainer.evaluate()
print(results)
```
## License
This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for more details.
## Citation
If you use this model in your research, please cite it as follows:
```bibtex
@misc{TOSBert,
author = {Himanshu Mohanty},
title = {TOSBert: Fine-tuned BERT model for multi-label classification},
year = {2024},
publisher = {Hugging Face},
journal = {Hugging Face Model Hub},
howpublished = {\url{https://huggingface.co./CodeHima/TOSBert}}
}
```
## Acknowledgements
This project uses the [Hugging Face Transformers](https://huggingface.co./transformers/) library. Special thanks to the developers and contributors of this library.
``` |