README.md

---
license: apache-2.0
---

RoBERTa-based sentence-level classification model.
Similar to alex2awesome/quote-detection__roberta-base-sentence, except it's trained on a lot more data.

Scores this accuracy on the original gold-label training dataset mentioned in: https://arxiv.org/pdf/2305.14904.pdf 

{
    "eval_samples": 86,
    "test_f1": 0.8056872037914692,
    "test_loss": 0.4573868425250922,
}

{
"DIRECT QUOTE_f1": 0.9284126278345932, 
"full_f1": 0.8055235903337169, 
"BACKGROUND_f1": 0.770718232044199, 
"NO QUOTE_f1": 0.0, 
"INDIRECT QUOTE_f1": 0.8142250530785562, 
"PUBLISHED WORK_f1": 0.8680851063829788, 
"STATEMENT_f1": 0.9477124183006536, 
"PRESS REPORT_f1": 0.9457364341085273,
"DECLINED COMMENT_f1": 0.9444444444444444, 
"SOCIAL MEDIA POST_f1": 0.8936170212765957, 
"PROPOSAL/ORDER/LAW_f1": 0.5616438356164384, 
"PRICE SIGNAL_f1": 0.7272727272727273, 
"NARRATIVE_f1": 0.8833333333333333, 
"DIRECT OBSERVATION_f1": 0.4324324324324324, 
"COMMUNICATION_f1": 0.9672131147540983,
"PUBLIC SPEECH_f1": 0.983050847457627, 
"VOTE/POLL_f1": 0.8205128205128205,
"COURT PROCEEDING_f1": 0.9491525423728813
}