nyt-ingredient-tagger-jina-embeddings-v2-small-en

This model is a fine-tuned version of jinaai/jina-embeddings-v2-small-en on the nyt_ingredients dataset. It achieves the following results on the evaluation set:

• Loss: 0.9890
• Comment: {'precision': 0.4891238056515552, 'recall': 0.6700083542188805, 'f1': 0.5654524089306698, 'number': 7182}
• Name: {'precision': 0.7393011781290907, 'recall': 0.7889533634214485, 'f1': 0.7633206840983521, 'number': 9306}
• Qty: {'precision': 0.9253731343283582, 'recall': 0.9613688009624382, 'f1': 0.943027601127647, 'number': 7481}
• Range End: {'precision': 0.5454545454545454, 'recall': 0.5121951219512195, 'f1': 0.5283018867924528, 'number': 82}
• Unit: {'precision': 0.9031507061927674, 'recall': 0.9693486590038314, 'f1': 0.9350795436284751, 'number': 6003}
• Overall Precision: 0.7401
• Overall Recall: 0.8387
• Overall F1: 0.7863
• Overall Accuracy: 0.7817

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5
• label_smoothing_factor: 0.1

Training results

Training Loss	Epoch	Step	Validation Loss	Comment	Name	Qty	Range End	Unit	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.1585	0.2	1000	1.1247	{'precision': 0.38455309241826097, 'recall': 0.5557343475716794, 'f1': 0.454561770864493, 'number': 6836}	{'precision': 0.6500338458563002, 'recall': 0.7587763855965685, 'f1': 0.7002083333333333, 'number': 8859}	{'precision': 0.8947789025039957, 'recall': 0.9468639887244539, 'f1': 0.9200849140587551, 'number': 7095}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 74}	{'precision': 0.8575376112987412, 'recall': 0.9760615062030403, 'f1': 0.912968864917872, 'number': 5723}	0.6666	0.7984	0.7266	0.7220
1.1018	0.4	2000	1.0677	{'precision': 0.3960273712222011, 'recall': 0.6095669982445875, 'f1': 0.48012443829934326, 'number': 6836}	{'precision': 0.7094423320659062, 'recall': 0.7582119878090078, 'f1': 0.733016860369946, 'number': 8859}	{'precision': 0.8986009327115256, 'recall': 0.9505285412262157, 'f1': 0.9238356164383562, 'number': 7095}	{'precision': 0.2, 'recall': 0.02702702702702703, 'f1': 0.047619047619047616, 'number': 74}	{'precision': 0.8794009877329935, 'recall': 0.9645290931329722, 'f1': 0.9199999999999999, 'number': 5723}	0.6853	0.8098	0.7424	0.7415
1.0676	0.59	3000	1.0472	{'precision': 0.41734173417341736, 'recall': 0.6104447045055588, 'f1': 0.4957528957528958, 'number': 6836}	{'precision': 0.7021688219122288, 'recall': 0.7784174286036799, 'f1': 0.7383297644539616, 'number': 8859}	{'precision': 0.8949375410913872, 'recall': 0.9592670894996477, 'f1': 0.9259863945578231, 'number': 7095}	{'precision': 0.5161290322580645, 'recall': 0.21621621621621623, 'f1': 0.3047619047619048, 'number': 74}	{'precision': 0.8788844621513944, 'recall': 0.9636554254761489, 'f1': 0.9193198866477745, 'number': 5723}	0.6939	0.8188	0.7512	0.7541
1.0613	0.79	4000	1.0459	{'precision': 0.4413024850042845, 'recall': 0.6026916325336454, 'f1': 0.5095226317091268, 'number': 6836}	{'precision': 0.7297499465697799, 'recall': 0.7708544982503669, 'f1': 0.7497392545424604, 'number': 8859}	{'precision': 0.9064651100013497, 'recall': 0.9465821000704722, 'f1': 0.9260893546607832, 'number': 7095}	{'precision': 0.34210526315789475, 'recall': 0.17567567567567569, 'f1': 0.23214285714285715, 'number': 74}	{'precision': 0.8965631196298744, 'recall': 0.9481041411846933, 'f1': 0.9216135881104034, 'number': 5723}	0.7177	0.8082	0.7603	0.7502
1.045	0.99	5000	1.0292	{'precision': 0.43983577218654596, 'recall': 0.6111761263897015, 'f1': 0.5115396388123661, 'number': 6836}	{'precision': 0.7188987787207618, 'recall': 0.7840614064792866, 'f1': 0.7500674909562118, 'number': 8859}	{'precision': 0.886005680351149, 'recall': 0.9673009161381254, 'f1': 0.9248702917593155, 'number': 7095}	{'precision': 0.3541666666666667, 'recall': 0.22972972972972974, 'f1': 0.27868852459016397, 'number': 74}	{'precision': 0.8777340676632572, 'recall': 0.974663637952123, 'f1': 0.9236628580890875, 'number': 5723}	0.7080	0.8249	0.7620	0.7610
1.0334	1.19	6000	1.0344	{'precision': 0.47399084477736164, 'recall': 0.6664716208308953, 'f1': 0.5539883268482491, 'number': 6836}	{'precision': 0.7198329853862213, 'recall': 0.7784174286036799, 'f1': 0.7479798253701395, 'number': 8859}	{'precision': 0.9296510806611104, 'recall': 0.927554615926709, 'f1': 0.92860166502046, 'number': 7095}	{'precision': 0.36666666666666664, 'recall': 0.2972972972972973, 'f1': 0.3283582089552239, 'number': 74}	{'precision': 0.8982691051600261, 'recall': 0.9612091560370435, 'f1': 0.9286739258884106, 'number': 5723}	0.7258	0.8240	0.7718	0.7595
1.0187	1.39	7000	1.0210	{'precision': 0.4423198816818086, 'recall': 0.6124926857811586, 'f1': 0.5136793031529874, 'number': 6836}	{'precision': 0.7155410238070911, 'recall': 0.7904955412574782, 'f1': 0.751153062318996, 'number': 8859}	{'precision': 0.8767850372804247, 'recall': 0.9778717406624383, 'f1': 0.9245735607675907, 'number': 7095}	{'precision': 0.37142857142857144, 'recall': 0.17567567567567569, 'f1': 0.23853211009174313, 'number': 74}	{'precision': 0.8888354957552459, 'recall': 0.9695963655425476, 'f1': 0.927461139896373, 'number': 5723}	0.7083	0.8287	0.7638	0.7651
1.0319	1.58	8000	1.0136	{'precision': 0.46955690149824675, 'recall': 0.6464306612053833, 'f1': 0.5439773496645535, 'number': 6836}	{'precision': 0.7399957428693061, 'recall': 0.7848515633818716, 'f1': 0.7617639003012875, 'number': 8859}	{'precision': 0.8963893249607535, 'recall': 0.9657505285412262, 'f1': 0.9297781396295542, 'number': 7095}	{'precision': 0.45098039215686275, 'recall': 0.3108108108108108, 'f1': 0.368, 'number': 74}	{'precision': 0.8991981672394044, 'recall': 0.9601607548488555, 'f1': 0.9286800743620078, 'number': 5723}	0.7280	0.8305	0.7759	0.7700
1.0154	1.78	9000	1.0071	{'precision': 0.47295907875796833, 'recall': 0.6729081334113517, 'f1': 0.5554884675763797, 'number': 6836}	{'precision': 0.7443054218800128, 'recall': 0.7856417202844564, 'f1': 0.7644151565074134, 'number': 8859}	{'precision': 0.9104236718224613, 'recall': 0.9540521494009866, 'f1': 0.9317274604267033, 'number': 7095}	{'precision': 0.4126984126984127, 'recall': 0.35135135135135137, 'f1': 0.3795620437956204, 'number': 74}	{'precision': 0.8885522959183674, 'recall': 0.9737899702952997, 'f1': 0.9292205085452273, 'number': 5723}	0.7285	0.8370	0.7790	0.7732
1.011	1.98	10000	1.0127	{'precision': 0.4703654417033473, 'recall': 0.6721767115272089, 'f1': 0.5534477566997892, 'number': 6836}	{'precision': 0.742723104808615, 'recall': 0.7863189976295293, 'f1': 0.7638995503892971, 'number': 8859}	{'precision': 0.915743991358358, 'recall': 0.9558844256518675, 'f1': 0.9353837666367836, 'number': 7095}	{'precision': 0.39285714285714285, 'recall': 0.2972972972972973, 'f1': 0.3384615384615385, 'number': 74}	{'precision': 0.901291060630822, 'recall': 0.9636554254761489, 'f1': 0.9314305016044586, 'number': 5723}	0.7296	0.8353	0.7789	0.7712
0.9958	2.18	11000	1.0024	{'precision': 0.4715127701375246, 'recall': 0.6670567583382094, 'f1': 0.5524928818077179, 'number': 6836}	{'precision': 0.7483029845921776, 'recall': 0.7839485269217744, 'f1': 0.7657111356119073, 'number': 8859}	{'precision': 0.9206457791763579, 'recall': 0.9484143763213531, 'f1': 0.9343237989447376, 'number': 7095}	{'precision': 0.35, 'recall': 0.3783783783783784, 'f1': 0.36363636363636365, 'number': 74}	{'precision': 0.8947876447876448, 'recall': 0.9718679014502883, 'f1': 0.9317363263254879, 'number': 5723}	0.7318	0.8334	0.7793	0.7763
1.0042	2.38	12000	1.0007	{'precision': 0.4789602641951635, 'recall': 0.657694558221182, 'f1': 0.5542747950440732, 'number': 6836}	{'precision': 0.7298981923955953, 'recall': 0.7930917710802574, 'f1': 0.7601839329185826, 'number': 8859}	{'precision': 0.9267211525141986, 'recall': 0.9429175475687104, 'f1': 0.9347491965907503, 'number': 7095}	{'precision': 0.35555555555555557, 'recall': 0.43243243243243246, 'f1': 0.3902439024390244, 'number': 74}	{'precision': 0.9089250165892502, 'recall': 0.9573650183470208, 'f1': 0.932516381584546, 'number': 5723}	0.7333	0.8299	0.7786	0.7769
1.0048	2.57	13000	0.9943	{'precision': 0.47168994262206343, 'recall': 0.6373610298420129, 'f1': 0.542151434082001, 'number': 6836}	{'precision': 0.7342649994746243, 'recall': 0.7888023478947963, 'f1': 0.7605572485851111, 'number': 8859}	{'precision': 0.8968563263185243, 'recall': 0.9730796335447498, 'f1': 0.9334144527817211, 'number': 7095}	{'precision': 0.4307692307692308, 'recall': 0.3783783783783784, 'f1': 0.4028776978417266, 'number': 74}	{'precision': 0.8915373540233562, 'recall': 0.9737899702952997, 'f1': 0.9308501753799899, 'number': 5723}	0.7278	0.8343	0.7774	0.7787
0.9911	2.77	14000	0.9951	{'precision': 0.4768817204301075, 'recall': 0.6487712112346401, 'f1': 0.5497025285076846, 'number': 6836}	{'precision': 0.729257190151045, 'recall': 0.7956880009030365, 'f1': 0.7610256410256411, 'number': 8859}	{'precision': 0.9042483230303827, 'recall': 0.9689922480620154, 'f1': 0.9355014287658184, 'number': 7095}	{'precision': 0.5319148936170213, 'recall': 0.33783783783783783, 'f1': 0.4132231404958678, 'number': 74}	{'precision': 0.8965628529933839, 'recall': 0.9708195002621003, 'f1': 0.9322147651006711, 'number': 5723}	0.7296	0.8374	0.7798	0.7786
0.9991	2.97	15000	0.9921	{'precision': 0.4791033832617576, 'recall': 0.6691047396138092, 'f1': 0.5583836904107916, 'number': 6836}	{'precision': 0.7481054541573273, 'recall': 0.7911728186025511, 'f1': 0.7690366469168313, 'number': 8859}	{'precision': 0.9137861466039005, 'recall': 0.9575757575757575, 'f1': 0.935168616655196, 'number': 7095}	{'precision': 0.4166666666666667, 'recall': 0.40540540540540543, 'f1': 0.4109589041095891, 'number': 74}	{'precision': 0.8894720101781171, 'recall': 0.977284640922593, 'f1': 0.9313129631171426, 'number': 5723}	0.7337	0.8395	0.7831	0.7807
0.9805	3.17	16000	0.9880	{'precision': 0.4859154929577465, 'recall': 0.6560854300760679, 'f1': 0.5583219220714553, 'number': 6836}	{'precision': 0.7423922231614539, 'recall': 0.7930917710802574, 'f1': 0.7669049828084921, 'number': 8859}	{'precision': 0.9187102018696653, 'recall': 0.9557434813248766, 'f1': 0.9368610113290964, 'number': 7095}	{'precision': 0.4444444444444444, 'recall': 0.3783783783783784, 'f1': 0.4087591240875913, 'number': 74}	{'precision': 0.9000486775920817, 'recall': 0.9692468984798183, 'f1': 0.9333669863705198, 'number': 5723}	0.7389	0.8349	0.7840	0.7822
0.9848	3.37	17000	0.9842	{'precision': 0.48933174482833314, 'recall': 0.6609128145114102, 'f1': 0.5623249735515589, 'number': 6836}	{'precision': 0.7466623945316672, 'recall': 0.7891409865673327, 'f1': 0.7673142355394577, 'number': 8859}	{'precision': 0.9149737656397148, 'recall': 0.9585623678646934, 'f1': 0.936261013215859, 'number': 7095}	{'precision': 0.4126984126984127, 'recall': 0.35135135135135137, 'f1': 0.3795620437956204, 'number': 74}	{'precision': 0.899171943497321, 'recall': 0.9676742966975362, 'f1': 0.9321663019693655, 'number': 5723}	0.7403	0.8351	0.7848	0.7824
0.9771	3.56	18000	0.9834	{'precision': 0.4883396023643203, 'recall': 0.6647162083089526, 'f1': 0.5630382256365777, 'number': 6836}	{'precision': 0.7373874816830647, 'recall': 0.795236482672988, 'f1': 0.7652202248411449, 'number': 8859}	{'precision': 0.9162388543636855, 'recall': 0.9558844256518675, 'f1': 0.9356418569359177, 'number': 7095}	{'precision': 0.4583333333333333, 'recall': 0.44594594594594594, 'f1': 0.4520547945205479, 'number': 74}	{'precision': 0.8992864093415505, 'recall': 0.968897431417089, 'f1': 0.9327950206072841, 'number': 5723}	0.7369	0.8378	0.7841	0.7837
0.9787	3.76	19000	0.9832	{'precision': 0.4892808110676946, 'recall': 0.677735517846694, 'f1': 0.5682919349892671, 'number': 6836}	{'precision': 0.7466029723991507, 'recall': 0.7938819279828423, 'f1': 0.7695169319984682, 'number': 8859}	{'precision': 0.9206090266449157, 'recall': 0.9544749823819592, 'f1': 0.9372361774271676, 'number': 7095}	{'precision': 0.4189189189189189, 'recall': 0.4189189189189189, 'f1': 0.4189189189189189, 'number': 74}	{'precision': 0.9048244174597965, 'recall': 0.9634806919447843, 'f1': 0.9332317847169331, 'number': 5723}	0.7399	0.8389	0.7863	0.7844
0.9746	3.96	20000	0.9827	{'precision': 0.4950890447922288, 'recall': 0.6710064365125804, 'f1': 0.5697782746413266, 'number': 6836}	{'precision': 0.7460368124268539, 'recall': 0.7915114572750874, 'f1': 0.768101654069449, 'number': 8859}	{'precision': 0.9120629837203096, 'recall': 0.9633544749823819, 'f1': 0.9370073342929603, 'number': 7095}	{'precision': 0.40963855421686746, 'recall': 0.4594594594594595, 'f1': 0.43312101910828027, 'number': 74}	{'precision': 0.9003893575600259, 'recall': 0.9697710990739122, 'f1': 0.9337932194834694, 'number': 5723}	0.7412	0.8402	0.7876	0.7846
0.976	4.16	21000	0.9836	{'precision': 0.4884607241160279, 'recall': 0.6749561146869514, 'f1': 0.5667608401916225, 'number': 6836}	{'precision': 0.7483774869666986, 'recall': 0.7939948075403545, 'f1': 0.7705115565779385, 'number': 8859}	{'precision': 0.9147193123824873, 'recall': 0.9599718111346018, 'f1': 0.936799394814662, 'number': 7095}	{'precision': 0.4125, 'recall': 0.44594594594594594, 'f1': 0.42857142857142855, 'number': 74}	{'precision': 0.9035317200784827, 'recall': 0.9655774943211602, 'f1': 0.9335247909451813, 'number': 5723}	0.7393	0.8402	0.7865	0.7854
0.9635	4.36	22000	0.9832	{'precision': 0.49533612093920876, 'recall': 0.6758338209479228, 'f1': 0.5716760502381983, 'number': 6836}	{'precision': 0.7475583864118895, 'recall': 0.7948978440004515, 'f1': 0.7705016685814322, 'number': 8859}	{'precision': 0.9186408555570597, 'recall': 0.9564482029598309, 'f1': 0.9371633752244165, 'number': 7095}	{'precision': 0.391304347826087, 'recall': 0.4864864864864865, 'f1': 0.43373493975903615, 'number': 74}	{'precision': 0.9026418786692759, 'recall': 0.9671500961034423, 'f1': 0.9337832138338253, 'number': 5723}	0.7423	0.8402	0.7882	0.7851
0.9688	4.55	23000	0.9836	{'precision': 0.4930739135032251, 'recall': 0.6821240491515506, 'f1': 0.5723930522310194, 'number': 6836}	{'precision': 0.7467897697124058, 'recall': 0.7943334462128908, 'f1': 0.7698282463625424, 'number': 8859}	{'precision': 0.9208593962469405, 'recall': 0.9544749823819592, 'f1': 0.9373659076752716, 'number': 7095}	{'precision': 0.35714285714285715, 'recall': 0.5405405405405406, 'f1': 0.4301075268817204, 'number': 74}	{'precision': 0.9021013194331324, 'recall': 0.9676742966975362, 'f1': 0.9337379868487607, 'number': 5723}	0.7403	0.8413	0.7876	0.7860
0.9669	4.75	24000	0.9803	{'precision': 0.4968897468897469, 'recall': 0.677735517846694, 'f1': 0.5733910891089109, 'number': 6836}	{'precision': 0.7478168264110756, 'recall': 0.7926402528502088, 'f1': 0.7695764151460354, 'number': 8859}	{'precision': 0.9173631706659477, 'recall': 0.9591261451726568, 'f1': 0.9377799214497348, 'number': 7095}	{'precision': 0.4, 'recall': 0.4864864864864865, 'f1': 0.43902439024390244, 'number': 74}	{'precision': 0.8998864189518092, 'recall': 0.9690721649484536, 'f1': 0.9331987211845869, 'number': 5723}	0.7424	0.8410	0.7886	0.7873
0.9691	4.95	25000	0.9796	{'precision': 0.4962978860392746, 'recall': 0.6765652428320655, 'f1': 0.5725781491798204, 'number': 6836}	{'precision': 0.7474457215836526, 'recall': 0.7927531324077209, 'f1': 0.7694330320460149, 'number': 8859}	{'precision': 0.9183783783783783, 'recall': 0.9578576462297392, 'f1': 0.9377026560883062, 'number': 7095}	{'precision': 0.4065934065934066, 'recall': 0.5, 'f1': 0.4484848484848485, 'number': 74}	{'precision': 0.9002599090318388, 'recall': 0.968373230822995, 'f1': 0.933075174678003, 'number': 5723}	0.7423	0.8403	0.7883	0.7871

Framework versions

• Transformers 4.34.1
• Pytorch 2.1.0+cu118
• Datasets 2.14.6
• Tokenizers 0.14.1