|
--- |
|
license: mit |
|
base_model: cardiffnlp/twitter-roberta-base-2019-90m |
|
tags: |
|
- generated_from_trainer |
|
model-index: |
|
- name: 2020-Q3-75p-filtered |
|
results: [] |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# 2020-Q3-75p-filtered |
|
|
|
This model is a fine-tuned version of [cardiffnlp/twitter-roberta-base-2019-90m](https://huggingface.co./cardiffnlp/twitter-roberta-base-2019-90m) on an unknown dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 3.1267 |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 4.1e-07 |
|
- train_batch_size: 16 |
|
- eval_batch_size: 16 |
|
- seed: 42 |
|
- optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08 |
|
- lr_scheduler_type: linear |
|
- training_steps: 2400000 |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | |
|
|:-------------:|:-----:|:-------:|:---------------:| |
|
| No log | 0.07 | 8000 | 3.4513 | |
|
| 3.6645 | 0.14 | 16000 | 3.3790 | |
|
| 3.6645 | 0.2 | 24000 | 3.3226 | |
|
| 3.4678 | 0.27 | 32000 | 3.2901 | |
|
| 3.4678 | 0.34 | 40000 | 3.2590 | |
|
| 3.4118 | 0.41 | 48000 | 3.2534 | |
|
| 3.4118 | 0.47 | 56000 | 3.2357 | |
|
| 3.3843 | 0.54 | 64000 | 3.2228 | |
|
| 3.3843 | 0.61 | 72000 | 3.2331 | |
|
| 3.3633 | 0.68 | 80000 | 3.2047 | |
|
| 3.3633 | 0.74 | 88000 | 3.2138 | |
|
| 3.3474 | 0.81 | 96000 | 3.2050 | |
|
| 3.3474 | 0.88 | 104000 | 3.2051 | |
|
| 3.3414 | 0.95 | 112000 | 3.1930 | |
|
| 3.3414 | 1.01 | 120000 | 3.2002 | |
|
| 3.335 | 1.08 | 128000 | 3.1920 | |
|
| 3.335 | 1.15 | 136000 | 3.1914 | |
|
| 3.3283 | 1.22 | 144000 | 3.1853 | |
|
| 3.3283 | 1.28 | 152000 | 3.1825 | |
|
| 3.3276 | 1.35 | 160000 | 3.1827 | |
|
| 3.3276 | 1.42 | 168000 | 3.1756 | |
|
| 3.323 | 1.49 | 176000 | 3.1865 | |
|
| 3.323 | 1.56 | 184000 | 3.1749 | |
|
| 3.3275 | 1.62 | 192000 | 3.1782 | |
|
| 3.3275 | 1.69 | 200000 | 3.1676 | |
|
| 3.3309 | 1.76 | 208000 | 3.1832 | |
|
| 3.3309 | 1.83 | 216000 | 3.1744 | |
|
| 3.3166 | 1.89 | 224000 | 3.1645 | |
|
| 3.3166 | 1.96 | 232000 | 3.1770 | |
|
| 3.3206 | 2.03 | 240000 | 3.1656 | |
|
| 3.3206 | 2.1 | 248000 | 3.1561 | |
|
| 3.3228 | 2.16 | 256000 | 3.1665 | |
|
| 3.3228 | 2.23 | 264000 | 3.1657 | |
|
| 3.3208 | 2.3 | 272000 | 3.1693 | |
|
| 3.3208 | 2.37 | 280000 | 3.1778 | |
|
| 3.3106 | 2.43 | 288000 | 3.1760 | |
|
| 3.3106 | 2.5 | 296000 | 3.1664 | |
|
| 3.3189 | 2.57 | 304000 | 3.1677 | |
|
| 3.3189 | 2.64 | 312000 | 3.1600 | |
|
| 3.319 | 2.7 | 320000 | 3.1570 | |
|
| 3.319 | 2.77 | 328000 | 3.1699 | |
|
| 3.3236 | 2.84 | 336000 | 3.1578 | |
|
| 3.3236 | 2.91 | 344000 | 3.1665 | |
|
| 3.3205 | 2.98 | 352000 | 3.1558 | |
|
| 3.3205 | 3.04 | 360000 | 3.1678 | |
|
| 3.3114 | 3.11 | 368000 | 3.1597 | |
|
| 3.3114 | 3.18 | 376000 | 3.1618 | |
|
| 3.3067 | 3.25 | 384000 | 3.1584 | |
|
| 3.3067 | 3.31 | 392000 | 3.1597 | |
|
| 3.314 | 3.38 | 400000 | 3.1565 | |
|
| 3.314 | 3.45 | 408000 | 3.1612 | |
|
| 3.3183 | 3.52 | 416000 | 3.1637 | |
|
| 3.3183 | 3.58 | 424000 | 3.1569 | |
|
| 3.318 | 3.65 | 432000 | 3.1576 | |
|
| 3.318 | 3.72 | 440000 | 3.1639 | |
|
| 3.3114 | 3.79 | 448000 | 3.1460 | |
|
| 3.3114 | 3.85 | 456000 | 3.1611 | |
|
| 3.3068 | 3.92 | 464000 | 3.1587 | |
|
| 3.3068 | 3.99 | 472000 | 3.1542 | |
|
| 3.3166 | 4.06 | 480000 | 3.1422 | |
|
| 3.3166 | 4.12 | 488000 | 3.1604 | |
|
| 3.3057 | 4.19 | 496000 | 3.1587 | |
|
| 3.3057 | 4.26 | 504000 | 3.1576 | |
|
| 3.3176 | 4.33 | 512000 | 3.1602 | |
|
| 3.3176 | 4.4 | 520000 | 3.1544 | |
|
| 3.3126 | 4.46 | 528000 | 3.1478 | |
|
| 3.3126 | 4.53 | 536000 | 3.1520 | |
|
| 3.3044 | 4.6 | 544000 | 3.1581 | |
|
| 3.3044 | 4.67 | 552000 | 3.1625 | |
|
| 3.3118 | 4.73 | 560000 | 3.1510 | |
|
| 3.3118 | 4.8 | 568000 | 3.1548 | |
|
| 3.3085 | 4.87 | 576000 | 3.1539 | |
|
| 3.3085 | 4.94 | 584000 | 3.1503 | |
|
| 3.3014 | 5.0 | 592000 | 3.1504 | |
|
| 3.3014 | 5.07 | 600000 | 3.1534 | |
|
| 3.3115 | 5.14 | 608000 | 3.1551 | |
|
| 3.3115 | 5.21 | 616000 | 3.1493 | |
|
| 3.3079 | 5.27 | 624000 | 3.1427 | |
|
| 3.3079 | 5.34 | 632000 | 3.1500 | |
|
| 3.3138 | 5.41 | 640000 | 3.1546 | |
|
| 3.3138 | 5.48 | 648000 | 3.1482 | |
|
| 3.3096 | 5.54 | 656000 | 3.1346 | |
|
| 3.3096 | 5.61 | 664000 | 3.1328 | |
|
| 3.3121 | 5.68 | 672000 | 3.1500 | |
|
| 3.3121 | 5.75 | 680000 | 3.1312 | |
|
| 3.3195 | 5.82 | 688000 | 3.1440 | |
|
| 3.3195 | 5.88 | 696000 | 3.1191 | |
|
| 3.3091 | 5.95 | 704000 | 3.1397 | |
|
| 3.3091 | 6.02 | 712000 | 3.1485 | |
|
| 3.3089 | 6.09 | 720000 | 3.1340 | |
|
| 3.3089 | 6.15 | 728000 | 3.1385 | |
|
| 3.3062 | 6.22 | 736000 | 3.1358 | |
|
| 3.3062 | 6.29 | 744000 | 3.1296 | |
|
| 3.3102 | 6.36 | 752000 | 3.1260 | |
|
| 3.3102 | 6.42 | 760000 | 3.1428 | |
|
| 3.3088 | 6.49 | 768000 | 3.1372 | |
|
| 3.3088 | 6.56 | 776000 | 3.1404 | |
|
| 3.3096 | 6.63 | 784000 | 3.1362 | |
|
| 3.3096 | 6.69 | 792000 | 3.1408 | |
|
| 3.3079 | 6.76 | 800000 | 3.1350 | |
|
| 3.3079 | 6.83 | 808000 | 3.1461 | |
|
| 3.3099 | 6.9 | 816000 | 3.1420 | |
|
| 3.3099 | 6.96 | 824000 | 3.1213 | |
|
| 3.3015 | 7.03 | 832000 | 3.1366 | |
|
| 3.3015 | 7.1 | 840000 | 3.1401 | |
|
| 3.3045 | 7.17 | 848000 | 3.1295 | |
|
| 3.3045 | 7.24 | 856000 | 3.1323 | |
|
| 3.3085 | 7.3 | 864000 | 3.1368 | |
|
| 3.3085 | 7.37 | 872000 | 3.1275 | |
|
| 3.3061 | 7.44 | 880000 | 3.1326 | |
|
| 3.3061 | 7.51 | 888000 | 3.1377 | |
|
| 3.309 | 7.57 | 896000 | 3.1407 | |
|
| 3.309 | 7.64 | 904000 | 3.1324 | |
|
| 3.3024 | 7.71 | 912000 | 3.1187 | |
|
| 3.3024 | 7.78 | 920000 | 3.1514 | |
|
| 3.2955 | 7.84 | 928000 | 3.1351 | |
|
| 3.2955 | 7.91 | 936000 | 3.1308 | |
|
| 3.3122 | 7.98 | 944000 | 3.1405 | |
|
| 3.3122 | 8.05 | 952000 | 3.1291 | |
|
| 3.304 | 8.11 | 960000 | 3.1244 | |
|
| 3.304 | 8.18 | 968000 | 3.1409 | |
|
| 3.3046 | 8.25 | 976000 | 3.1355 | |
|
| 3.3046 | 8.32 | 984000 | 3.1416 | |
|
| 3.3022 | 8.38 | 992000 | 3.1258 | |
|
| 3.3022 | 8.45 | 1000000 | 3.1332 | |
|
| 3.3004 | 8.52 | 1008000 | 3.1430 | |
|
| 3.3004 | 8.59 | 1016000 | 3.1282 | |
|
| 3.3045 | 8.66 | 1024000 | 3.1287 | |
|
| 3.3045 | 8.72 | 1032000 | 3.1368 | |
|
| 3.3047 | 8.79 | 1040000 | 3.1362 | |
|
| 3.3047 | 8.86 | 1048000 | 3.1268 | |
|
| 3.3044 | 8.93 | 1056000 | 3.1329 | |
|
| 3.3044 | 8.99 | 1064000 | 3.1245 | |
|
| 3.2961 | 9.06 | 1072000 | 3.1271 | |
|
| 3.2961 | 9.13 | 1080000 | 3.1300 | |
|
| 3.2999 | 9.2 | 1088000 | 3.1369 | |
|
| 3.2999 | 9.26 | 1096000 | 3.1425 | |
|
| 3.3012 | 9.33 | 1104000 | 3.1213 | |
|
| 3.3012 | 9.4 | 1112000 | 3.1285 | |
|
| 3.3008 | 9.47 | 1120000 | 3.1353 | |
|
| 3.3008 | 9.53 | 1128000 | 3.1367 | |
|
| 3.3028 | 9.6 | 1136000 | 3.1294 | |
|
| 3.3028 | 9.67 | 1144000 | 3.1340 | |
|
| 3.3043 | 9.74 | 1152000 | 3.1330 | |
|
| 3.3043 | 9.8 | 1160000 | 3.1380 | |
|
| 3.2976 | 9.87 | 1168000 | 3.1198 | |
|
| 3.2976 | 9.94 | 1176000 | 3.1290 | |
|
| 3.3048 | 10.01 | 1184000 | 3.1458 | |
|
| 3.3048 | 10.08 | 1192000 | 3.1274 | |
|
| 3.3038 | 10.14 | 1200000 | 3.1181 | |
|
| 3.3038 | 10.21 | 1208000 | 3.1279 | |
|
| 3.3066 | 10.28 | 1216000 | 3.1230 | |
|
| 3.3066 | 10.35 | 1224000 | 3.1320 | |
|
| 3.3019 | 10.41 | 1232000 | 3.1203 | |
|
| 3.3019 | 10.48 | 1240000 | 3.1349 | |
|
| 3.3037 | 10.55 | 1248000 | 3.1323 | |
|
| 3.3037 | 10.62 | 1256000 | 3.1343 | |
|
| 3.2868 | 10.68 | 1264000 | 3.1262 | |
|
| 3.2868 | 10.75 | 1272000 | 3.1266 | |
|
| 3.3033 | 10.82 | 1280000 | 3.1283 | |
|
| 3.3033 | 10.89 | 1288000 | 3.1290 | |
|
| 3.2984 | 10.95 | 1296000 | 3.1177 | |
|
| 3.2984 | 11.02 | 1304000 | 3.1234 | |
|
| 3.2982 | 11.09 | 1312000 | 3.1310 | |
|
| 3.2982 | 11.16 | 1320000 | 3.1409 | |
|
| 3.303 | 11.23 | 1328000 | 3.1330 | |
|
| 3.303 | 11.29 | 1336000 | 3.1281 | |
|
| 3.2976 | 11.36 | 1344000 | 3.1286 | |
|
| 3.2976 | 11.43 | 1352000 | 3.1283 | |
|
| 3.2923 | 11.5 | 1360000 | 3.1146 | |
|
| 3.2923 | 11.56 | 1368000 | 3.1387 | |
|
| 3.2988 | 11.63 | 1376000 | 3.1278 | |
|
| 3.2988 | 11.7 | 1384000 | 3.1225 | |
|
| 3.299 | 11.77 | 1392000 | 3.1341 | |
|
| 3.299 | 11.83 | 1400000 | 3.1211 | |
|
| 3.2993 | 11.9 | 1408000 | 3.1026 | |
|
| 3.2993 | 11.97 | 1416000 | 3.1223 | |
|
| 3.2942 | 12.04 | 1424000 | 3.1200 | |
|
| 3.2942 | 12.1 | 1432000 | 3.1246 | |
|
| 3.3062 | 12.17 | 1440000 | 3.1325 | |
|
| 3.3062 | 12.24 | 1448000 | 3.1388 | |
|
| 3.297 | 12.31 | 1456000 | 3.1371 | |
|
| 3.297 | 12.37 | 1464000 | 3.1272 | |
|
| 3.3033 | 12.44 | 1472000 | 3.1231 | |
|
| 3.3033 | 12.51 | 1480000 | 3.1316 | |
|
| 3.291 | 12.58 | 1488000 | 3.1393 | |
|
| 3.291 | 12.65 | 1496000 | 3.1269 | |
|
| 3.3054 | 12.71 | 1504000 | 3.1363 | |
|
| 3.3054 | 12.78 | 1512000 | 3.1249 | |
|
| 3.2908 | 12.85 | 1520000 | 3.1310 | |
|
| 3.2908 | 12.92 | 1528000 | 3.1213 | |
|
| 3.2987 | 12.98 | 1536000 | 3.1223 | |
|
| 3.2987 | 13.05 | 1544000 | 3.1134 | |
|
| 3.2965 | 13.12 | 1552000 | 3.1168 | |
|
| 3.2965 | 13.19 | 1560000 | 3.1230 | |
|
| 3.2931 | 13.25 | 1568000 | 3.1132 | |
|
| 3.2931 | 13.32 | 1576000 | 3.1196 | |
|
| 3.301 | 13.39 | 1584000 | 3.1287 | |
|
| 3.301 | 13.46 | 1592000 | 3.1145 | |
|
| 3.3004 | 13.52 | 1600000 | 3.1291 | |
|
| 3.3004 | 13.59 | 1608000 | 3.1145 | |
|
| 3.2992 | 13.66 | 1616000 | 3.1292 | |
|
| 3.2992 | 13.73 | 1624000 | 3.1248 | |
|
| 3.2974 | 13.79 | 1632000 | 3.1315 | |
|
| 3.2974 | 13.86 | 1640000 | 3.1112 | |
|
| 3.2993 | 13.93 | 1648000 | 3.1217 | |
|
| 3.2993 | 14.0 | 1656000 | 3.1362 | |
|
| 3.2934 | 14.07 | 1664000 | 3.1199 | |
|
| 3.2934 | 14.13 | 1672000 | 3.1276 | |
|
| 3.2964 | 14.2 | 1680000 | 3.1164 | |
|
| 3.2964 | 14.27 | 1688000 | 3.1172 | |
|
| 3.305 | 14.34 | 1696000 | 3.1320 | |
|
| 3.305 | 14.4 | 1704000 | 3.1269 | |
|
| 3.3022 | 14.47 | 1712000 | 3.1107 | |
|
| 3.3022 | 14.54 | 1720000 | 3.1097 | |
|
| 3.2969 | 14.61 | 1728000 | 3.1176 | |
|
| 3.2969 | 14.67 | 1736000 | 3.1282 | |
|
| 3.2976 | 14.74 | 1744000 | 3.1195 | |
|
| 3.2976 | 14.81 | 1752000 | 3.1154 | |
|
| 3.3004 | 14.88 | 1760000 | 3.1147 | |
|
| 3.3004 | 14.94 | 1768000 | 3.1094 | |
|
| 3.2908 | 15.01 | 1776000 | 3.1313 | |
|
| 3.2908 | 15.08 | 1784000 | 3.1280 | |
|
| 3.2896 | 15.15 | 1792000 | 3.1304 | |
|
| 3.2896 | 15.21 | 1800000 | 3.1329 | |
|
| 3.3061 | 15.28 | 1808000 | 3.1198 | |
|
| 3.3061 | 15.35 | 1816000 | 3.1258 | |
|
| 3.3056 | 15.42 | 1824000 | 3.1253 | |
|
| 3.3056 | 15.49 | 1832000 | 3.1200 | |
|
| 3.2921 | 15.55 | 1840000 | 3.1384 | |
|
| 3.2921 | 15.62 | 1848000 | 3.1225 | |
|
| 3.2895 | 15.69 | 1856000 | 3.1284 | |
|
| 3.2895 | 15.76 | 1864000 | 3.1201 | |
|
| 3.293 | 15.82 | 1872000 | 3.1256 | |
|
| 3.293 | 15.89 | 1880000 | 3.1166 | |
|
| 3.2963 | 15.96 | 1888000 | 3.1218 | |
|
| 3.2963 | 16.03 | 1896000 | 3.1193 | |
|
| 3.2908 | 16.09 | 1904000 | 3.1204 | |
|
| 3.2908 | 16.16 | 1912000 | 3.1325 | |
|
| 3.3039 | 16.23 | 1920000 | 3.1091 | |
|
| 3.3039 | 16.3 | 1928000 | 3.1250 | |
|
| 3.3011 | 16.36 | 1936000 | 3.1217 | |
|
| 3.3011 | 16.43 | 1944000 | 3.1208 | |
|
| 3.3003 | 16.5 | 1952000 | 3.1109 | |
|
| 3.3003 | 16.57 | 1960000 | 3.1252 | |
|
| 3.3012 | 16.63 | 1968000 | 3.1123 | |
|
| 3.3012 | 16.7 | 1976000 | 3.1213 | |
|
| 3.2885 | 16.77 | 1984000 | 3.1219 | |
|
| 3.2885 | 16.84 | 1992000 | 3.1254 | |
|
| 3.2982 | 16.91 | 2000000 | 3.1260 | |
|
| 3.2982 | 16.97 | 2008000 | 3.1167 | |
|
| 3.2962 | 17.04 | 2016000 | 3.1082 | |
|
| 3.2962 | 17.11 | 2024000 | 3.1204 | |
|
| 3.2889 | 17.18 | 2032000 | 3.1236 | |
|
| 3.2889 | 17.24 | 2040000 | 3.1325 | |
|
| 3.2892 | 17.31 | 2048000 | 3.1200 | |
|
| 3.2892 | 17.38 | 2056000 | 3.1231 | |
|
| 3.3028 | 17.45 | 2064000 | 3.1202 | |
|
| 3.3028 | 17.51 | 2072000 | 3.1189 | |
|
| 3.2889 | 17.58 | 2080000 | 3.1337 | |
|
| 3.2889 | 17.65 | 2088000 | 3.1156 | |
|
| 3.2985 | 17.72 | 2096000 | 3.1258 | |
|
| 3.2985 | 17.78 | 2104000 | 3.1358 | |
|
| 3.2949 | 17.85 | 2112000 | 3.1271 | |
|
| 3.2949 | 17.92 | 2120000 | 3.1250 | |
|
| 3.2987 | 17.99 | 2128000 | 3.1244 | |
|
| 3.2987 | 18.05 | 2136000 | 3.1221 | |
|
| 3.2884 | 18.12 | 2144000 | 3.1198 | |
|
| 3.2884 | 18.19 | 2152000 | 3.1170 | |
|
| 3.2918 | 18.26 | 2160000 | 3.1159 | |
|
| 3.2918 | 18.33 | 2168000 | 3.1153 | |
|
| 3.2995 | 18.39 | 2176000 | 3.1203 | |
|
| 3.2995 | 18.46 | 2184000 | 3.1107 | |
|
| 3.3003 | 18.53 | 2192000 | 3.1212 | |
|
| 3.3003 | 18.6 | 2200000 | 3.1330 | |
|
| 3.2921 | 18.66 | 2208000 | 3.1160 | |
|
| 3.2921 | 18.73 | 2216000 | 3.1192 | |
|
| 3.293 | 18.8 | 2224000 | 3.1164 | |
|
| 3.293 | 18.87 | 2232000 | 3.1225 | |
|
| 3.2969 | 18.93 | 2240000 | 3.1243 | |
|
| 3.2969 | 19.0 | 2248000 | 3.1152 | |
|
| 3.2891 | 19.07 | 2256000 | 3.1323 | |
|
| 3.2891 | 19.14 | 2264000 | 3.1077 | |
|
| 3.2903 | 19.2 | 2272000 | 3.1348 | |
|
| 3.2903 | 19.27 | 2280000 | 3.1202 | |
|
| 3.2986 | 19.34 | 2288000 | 3.1220 | |
|
| 3.2986 | 19.41 | 2296000 | 3.1236 | |
|
| 3.293 | 19.47 | 2304000 | 3.1224 | |
|
| 3.293 | 19.54 | 2312000 | 3.1247 | |
|
| 3.299 | 19.61 | 2320000 | 3.1235 | |
|
| 3.299 | 19.68 | 2328000 | 3.1201 | |
|
| 3.2898 | 19.75 | 2336000 | 3.1163 | |
|
| 3.2898 | 19.81 | 2344000 | 3.1289 | |
|
| 3.2956 | 19.88 | 2352000 | 3.1198 | |
|
| 3.2956 | 19.95 | 2360000 | 3.1251 | |
|
| 3.2926 | 20.02 | 2368000 | 3.1087 | |
|
| 3.2926 | 20.08 | 2376000 | 3.1097 | |
|
| 3.2958 | 20.15 | 2384000 | 3.1262 | |
|
| 3.2958 | 20.22 | 2392000 | 3.1308 | |
|
| 3.2862 | 20.29 | 2400000 | 3.1129 | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.35.0.dev0 |
|
- Pytorch 2.0.1+cu117 |
|
- Datasets 2.14.5 |
|
- Tokenizers 0.14.0 |
|
|
