--- tags: - sentence-transformers - sentence-similarity - feature-extraction - generated_from_trainer - dataset_size:41479299 - loss:RZTKMatryoshka2dLoss base_model: intfloat/multilingual-e5-base widget: - source_sentence: 'query: ремешок для apple watch' sentences: - 'passage: Настенная вешалка Компанит В 1 Ольха' - 'passage: Браслет сетчатый Apple ремешок для Apple Watch 38mm/40mm миланское плетение серебряный' - 'passage: Серьги серебряные MAZZARINI JEWELRY Руно (949)' - source_sentence: 'query: дисплей xiaomi redmi 5a' sentences: - 'passage: Дисплеї для телефонів Xiaomi Країна реєстрації бренда Китай Діагональ, " 5 Роздільна здатність 1280х720 Колір Чорний Сумісність Xiaomi Клас якості High Copy' - 'passage: Плавки мужские Shepa 034 M Темно-синие (sh0045)' - 'passage: Паста Страна-производитель товара Украина Особенности Без сахара Особенности Безглютеновые Особенности Без соли Тип Арахисовая паста Вес 500 г Упаковка Стеклянная банка' - source_sentence: 'query: морозильная камера беко' sentences: - 'passage: Защитное стекло на iPhone 6/6s/7/8/ SE 2020 0.33 mm Прозрачное' - 'passage: Женские шорты GCA Размер 25 Цвет Синий' - 'passage: Морозильна камера Beko RFNE290L21W' - source_sentence: 'query: samsung s6' sentences: - 'passage: Наручні годинники Casio Гарантія 24 місяці офіційної гарантії від виробника Колекція G-Shock Механізм Кварцовий Тип Чоловічі Формат часу 12/24 год Вид циферблата Арабські цифри Форма корпусу Квадратна Матеріал корпусу Пластик Основний колір циферблата Сірий Водонепроникність 200 м Тип кріплення Ремінець Матеріал ремінця/браслета Полімер Підсвітка Електролюмінесцентна Колір ремінця/браслета Чорний Колір корпусу Чорний Функції Будильник Функції Світовий час Функції Секундомір Функції Таймер Тип циферблата Електронний' - 'passage: Аксесуари до смарт-годинників і трекерів Тип Ремінець Матеріал ремінця Фтореластомер Колір Lilac Призначення Смарт-годинник' - 'passage: Мобільні телефони Samsung Частота процесора 1.9 ГГц Діагональ екрана 5.1 Кількість ядер 8 Тип матриці AMOLED Вбудована пам''ять 32 ГБ Кількість мегапікселів фронтальної камери 5 Мп Клас Смартфони' - source_sentence: 'query: джинсовые шорты женские' sentences: - 'passage: Шорты джинсовые женские H&M 0612481-01 36 Темно-серые (KAY2000000788180)' - 'passage: Пудра Kiko Milano Страна-производитель товара Италия Вес 13.5 г Эффект Матирующая Эффект Фиксирующая' - 'passage: Ремешок MilaneseBand для Apple Watch 49 | 45 | 44 | 42 mm Black (UP46001)' pipeline_tag: sentence-similarity library_name: sentence-transformers metrics: - dot_accuracy_10 - dot_precision_10 - dot_recall_10 - dot_ndcg_10 - dot_mrr_10 - dot_map_60 - dot_accuracy_1 - dot_accuracy_3 - dot_accuracy_5 - dot_precision_1 - dot_precision_3 - dot_precision_5 - dot_recall_1 - dot_recall_3 - dot_recall_5 - dot_map_100 - dot_ndcg_1 - dot_mrr_1 model-index: - name: SentenceTransformer based on intfloat/multilingual-e5-base results: - task: type: rztkinformation-retrieval name: RZTKInformation Retrieval dataset: name: 'validation matryoshka dim 768 ' type: validation--matryoshka_dim-768-- metrics: - type: dot_accuracy_10 value: 0.4731333758774729 name: Dot Accuracy 10 - type: dot_precision_10 value: 0.06023080195703043 name: Dot Precision 10 - type: dot_recall_10 value: 0.4248598214577602 name: Dot Recall 10 - type: dot_ndcg_10 value: 0.26921828728278263 name: Dot Ndcg 10 - type: dot_mrr_10 value: 0.23144252489339734 name: Dot Mrr 10 - type: dot_map_60 value: 0.22893628708481442 name: Dot Map 60 - task: type: rztkinformation-retrieval name: RZTKInformation Retrieval dataset: name: bm full type: bm-full metrics: - type: dot_accuracy_1 value: 0.6325136612021858 name: Dot Accuracy 1 - type: dot_accuracy_3 value: 0.7454462659380692 name: Dot Accuracy 3 - type: dot_accuracy_5 value: 0.8010018214936248 name: Dot Accuracy 5 - type: dot_accuracy_10 value: 0.8656648451730419 name: Dot Accuracy 10 - type: dot_precision_1 value: 0.6325136612021858 name: Dot Precision 1 - type: dot_precision_3 value: 0.6212811171827565 name: Dot Precision 3 - type: dot_precision_5 value: 0.6060109289617486 name: Dot Precision 5 - type: dot_precision_10 value: 0.5620218579234972 name: Dot Precision 10 - type: dot_recall_1 value: 0.04407246382453946 name: Dot Recall 1 - type: dot_recall_3 value: 0.12640655239704984 name: Dot Recall 3 - type: dot_recall_5 value: 0.19893825637685827 name: Dot Recall 5 - type: dot_recall_10 value: 0.3427203554138508 name: Dot Recall 10 - type: dot_ndcg_10 value: 0.6019842949278745 name: Dot Ndcg 10 - type: dot_mrr_10 value: 0.6988097045132557 name: Dot Mrr 10 - type: dot_map_100 value: 0.5595201386694402 name: Dot Map 100 - task: type: rztkinformation-retrieval name: RZTKInformation Retrieval dataset: name: core uk title type: core-uk-title metrics: - type: dot_accuracy_1 value: 0.7874015748031497 name: Dot Accuracy 1 - type: dot_accuracy_3 value: 0.9238845144356955 name: Dot Accuracy 3 - type: dot_accuracy_5 value: 0.963254593175853 name: Dot Accuracy 5 - type: dot_accuracy_10 value: 0.9908136482939632 name: Dot Accuracy 10 - type: dot_precision_1 value: 0.7874015748031497 name: Dot Precision 1 - type: dot_precision_3 value: 0.71084864391951 name: Dot Precision 3 - type: dot_precision_5 value: 0.6251968503937008 name: Dot Precision 5 - type: dot_precision_10 value: 0.38910761154855644 name: Dot Precision 10 - type: dot_recall_1 value: 0.24099904178644335 name: Dot Recall 1 - type: dot_recall_3 value: 0.5665401199850019 name: Dot Recall 3 - type: dot_recall_5 value: 0.7758597883597883 name: Dot Recall 5 - type: dot_recall_10 value: 0.9339369037203682 name: Dot Recall 10 - type: dot_ndcg_10 value: 0.8559515168818833 name: Dot Ndcg 10 - type: dot_mrr_10 value: 0.8622583635378909 name: Dot Mrr 10 - type: dot_map_100 value: 0.8088073597566148 name: Dot Map 100 - task: type: rztkinformation-retrieval name: RZTKInformation Retrieval dataset: name: core ru title type: core-ru-title metrics: - type: dot_accuracy_1 value: 0.7887139107611548 name: Dot Accuracy 1 - type: dot_accuracy_3 value: 0.9238845144356955 name: Dot Accuracy 3 - type: dot_accuracy_5 value: 0.9698162729658792 name: Dot Accuracy 5 - type: dot_accuracy_10 value: 0.9908136482939632 name: Dot Accuracy 10 - type: dot_precision_1 value: 0.7887139107611548 name: Dot Precision 1 - type: dot_precision_3 value: 0.716097987751531 name: Dot Precision 3 - type: dot_precision_5 value: 0.6278215223097113 name: Dot Precision 5 - type: dot_precision_10 value: 0.38818897637795274 name: Dot Precision 10 - type: dot_recall_1 value: 0.24199058451026953 name: Dot Recall 1 - type: dot_recall_3 value: 0.5687184935216432 name: Dot Recall 3 - type: dot_recall_5 value: 0.7833812440111653 name: Dot Recall 5 - type: dot_recall_10 value: 0.9320079781693954 name: Dot Recall 10 - type: dot_ndcg_10 value: 0.8555312781782969 name: Dot Ndcg 10 - type: dot_mrr_10 value: 0.8609090530350372 name: Dot Mrr 10 - type: dot_map_100 value: 0.8100770161330401 name: Dot Map 100 - task: type: rztkinformation-retrieval name: RZTKInformation Retrieval dataset: name: core uk options type: core-uk-options metrics: - type: dot_accuracy_1 value: 0.6666666666666666 name: Dot Accuracy 1 - type: dot_accuracy_3 value: 0.8385826771653543 name: Dot Accuracy 3 - type: dot_accuracy_5 value: 0.8976377952755905 name: Dot Accuracy 5 - type: dot_accuracy_10 value: 0.9593175853018373 name: Dot Accuracy 10 - type: dot_precision_1 value: 0.6666666666666666 name: Dot Precision 1 - type: dot_precision_3 value: 0.6067366579177602 name: Dot Precision 3 - type: dot_precision_5 value: 0.5488188976377953 name: Dot Precision 5 - type: dot_precision_10 value: 0.36351706036745407 name: Dot Precision 10 - type: dot_recall_1 value: 0.19475482231387742 name: Dot Recall 1 - type: dot_recall_3 value: 0.46922415948006496 name: Dot Recall 3 - type: dot_recall_5 value: 0.6658110444527766 name: Dot Recall 5 - type: dot_recall_10 value: 0.8585051868516437 name: Dot Recall 10 - type: dot_ndcg_10 value: 0.7589808846879811 name: Dot Ndcg 10 - type: dot_mrr_10 value: 0.7658990542848814 name: Dot Mrr 10 - type: dot_map_100 value: 0.70424935283601 name: Dot Map 100 - task: type: rztkinformation-retrieval name: RZTKInformation Retrieval dataset: name: core ru options type: core-ru-options metrics: - type: dot_accuracy_1 value: 0.6640419947506562 name: Dot Accuracy 1 - type: dot_accuracy_3 value: 0.84251968503937 name: Dot Accuracy 3 - type: dot_accuracy_5 value: 0.9015748031496063 name: Dot Accuracy 5 - type: dot_accuracy_10 value: 0.9566929133858267 name: Dot Accuracy 10 - type: dot_precision_1 value: 0.6640419947506562 name: Dot Precision 1 - type: dot_precision_3 value: 0.6115485564304461 name: Dot Precision 3 - type: dot_precision_5 value: 0.5493438320209975 name: Dot Precision 5 - type: dot_precision_10 value: 0.3593175853018372 name: Dot Precision 10 - type: dot_recall_1 value: 0.19697485730950295 name: Dot Recall 1 - type: dot_recall_3 value: 0.4768112319293422 name: Dot Recall 3 - type: dot_recall_5 value: 0.6704021372328459 name: Dot Recall 5 - type: dot_recall_10 value: 0.8482174103237096 name: Dot Recall 10 - type: dot_ndcg_10 value: 0.7555720318794616 name: Dot Ndcg 10 - type: dot_mrr_10 value: 0.7663896179644208 name: Dot Mrr 10 - type: dot_map_100 value: 0.7051780094121903 name: Dot Map 100 - task: type: rztkinformation-retrieval name: RZTKInformation Retrieval dataset: name: options uk title type: options-uk-title metrics: - type: dot_accuracy_1 value: 0.8228155339805825 name: Dot Accuracy 1 - type: dot_accuracy_3 value: 0.9368932038834952 name: Dot Accuracy 3 - type: dot_accuracy_5 value: 0.9733009708737864 name: Dot Accuracy 5 - type: dot_accuracy_10 value: 0.9927184466019418 name: Dot Accuracy 10 - type: dot_precision_1 value: 0.8228155339805825 name: Dot Precision 1 - type: dot_precision_3 value: 0.7378640776699029 name: Dot Precision 3 - type: dot_precision_5 value: 0.5868932038834951 name: Dot Precision 5 - type: dot_precision_10 value: 0.33616504854368934 name: Dot Precision 10 - type: dot_recall_1 value: 0.2577901063337957 name: Dot Recall 1 - type: dot_recall_3 value: 0.6599976883957466 name: Dot Recall 3 - type: dot_recall_5 value: 0.8476595006934813 name: Dot Recall 5 - type: dot_recall_10 value: 0.9600294729542302 name: Dot Recall 10 - type: dot_ndcg_10 value: 0.8799039195105632 name: Dot Ndcg 10 - type: dot_mrr_10 value: 0.884285906919402 name: Dot Mrr 10 - type: dot_map_100 value: 0.8247309682311416 name: Dot Map 100 - task: type: rztkinformation-retrieval name: RZTKInformation Retrieval dataset: name: options ru title type: options-ru-title metrics: - type: dot_accuracy_1 value: 0.8276699029126213 name: Dot Accuracy 1 - type: dot_accuracy_3 value: 0.9466019417475728 name: Dot Accuracy 3 - type: dot_accuracy_5 value: 0.9757281553398058 name: Dot Accuracy 5 - type: dot_accuracy_10 value: 0.9927184466019418 name: Dot Accuracy 10 - type: dot_precision_1 value: 0.8276699029126213 name: Dot Precision 1 - type: dot_precision_3 value: 0.744336569579288 name: Dot Precision 3 - type: dot_precision_5 value: 0.5859223300970873 name: Dot Precision 5 - type: dot_precision_10 value: 0.3366504854368932 name: Dot Precision 10 - type: dot_recall_1 value: 0.2581946370781322 name: Dot Recall 1 - type: dot_recall_3 value: 0.6647306981044845 name: Dot Recall 3 - type: dot_recall_5 value: 0.8462031900138696 name: Dot Recall 5 - type: dot_recall_10 value: 0.9612430651872399 name: Dot Recall 10 - type: dot_ndcg_10 value: 0.8822938032699722 name: Dot Ndcg 10 - type: dot_mrr_10 value: 0.8892924564647865 name: Dot Mrr 10 - type: dot_map_100 value: 0.8268474912473482 name: Dot Map 100 - task: type: rztkinformation-retrieval name: RZTKInformation Retrieval dataset: name: options uk options type: options-uk-options metrics: - type: dot_accuracy_1 value: 0.6966019417475728 name: Dot Accuracy 1 - type: dot_accuracy_3 value: 0.8567961165048543 name: Dot Accuracy 3 - type: dot_accuracy_5 value: 0.912621359223301 name: Dot Accuracy 5 - type: dot_accuracy_10 value: 0.9660194174757282 name: Dot Accuracy 10 - type: dot_precision_1 value: 0.6966019417475728 name: Dot Precision 1 - type: dot_precision_3 value: 0.6359223300970874 name: Dot Precision 3 - type: dot_precision_5 value: 0.5067961165048543 name: Dot Precision 5 - type: dot_precision_10 value: 0.308252427184466 name: Dot Precision 10 - type: dot_recall_1 value: 0.21357489597780857 name: Dot Recall 1 - type: dot_recall_3 value: 0.566412390198798 name: Dot Recall 3 - type: dot_recall_5 value: 0.7312644475265835 name: Dot Recall 5 - type: dot_recall_10 value: 0.8752224919093851 name: Dot Recall 10 - type: dot_ndcg_10 value: 0.7777231689661694 name: Dot Ndcg 10 - type: dot_mrr_10 value: 0.7871301433194634 name: Dot Mrr 10 - type: dot_map_100 value: 0.7193179001782998 name: Dot Map 100 - task: type: rztkinformation-retrieval name: RZTKInformation Retrieval dataset: name: options ru options type: options-ru-options metrics: - type: dot_accuracy_1 value: 0.6966019417475728 name: Dot Accuracy 1 - type: dot_accuracy_3 value: 0.8640776699029126 name: Dot Accuracy 3 - type: dot_accuracy_5 value: 0.9199029126213593 name: Dot Accuracy 5 - type: dot_accuracy_10 value: 0.9563106796116505 name: Dot Accuracy 10 - type: dot_precision_1 value: 0.6966019417475728 name: Dot Precision 1 - type: dot_precision_3 value: 0.6229773462783172 name: Dot Precision 3 - type: dot_precision_5 value: 0.5077669902912622 name: Dot Precision 5 - type: dot_precision_10 value: 0.3065533980582524 name: Dot Precision 10 - type: dot_recall_1 value: 0.21448509015256587 name: Dot Recall 1 - type: dot_recall_3 value: 0.5556518723994451 name: Dot Recall 3 - type: dot_recall_5 value: 0.729787910309755 name: Dot Recall 5 - type: dot_recall_10 value: 0.8700040453074435 name: Dot Recall 10 - type: dot_ndcg_10 value: 0.773046764832658 name: Dot Ndcg 10 - type: dot_mrr_10 value: 0.7868989828941287 name: Dot Mrr 10 - type: dot_map_100 value: 0.7130448695375623 name: Dot Map 100 - task: type: rztkinformation-retrieval name: RZTKInformation Retrieval dataset: name: rusisms uk title type: rusisms-uk-title metrics: - type: dot_accuracy_1 value: 0.8307692307692308 name: Dot Accuracy 1 - type: dot_accuracy_3 value: 0.8846153846153846 name: Dot Accuracy 3 - type: dot_accuracy_5 value: 0.9153846153846154 name: Dot Accuracy 5 - type: dot_accuracy_10 value: 0.9384615384615385 name: Dot Accuracy 10 - type: dot_precision_1 value: 0.8307692307692308 name: Dot Precision 1 - type: dot_precision_3 value: 0.7820512820512819 name: Dot Precision 3 - type: dot_precision_5 value: 0.7353846153846153 name: Dot Precision 5 - type: dot_precision_10 value: 0.6315384615384615 name: Dot Precision 10 - type: dot_recall_1 value: 0.15919107183151676 name: Dot Recall 1 - type: dot_recall_3 value: 0.3475834537942623 name: Dot Recall 3 - type: dot_recall_5 value: 0.48375611103728305 name: Dot Recall 5 - type: dot_recall_10 value: 0.6878983684359447 name: Dot Recall 10 - type: dot_ndcg_10 value: 0.8362864316561219 name: Dot Ndcg 10 - type: dot_mrr_10 value: 0.8670604395604395 name: Dot Mrr 10 - type: dot_map_100 value: 0.8294579734441355 name: Dot Map 100 - task: type: rztkinformation-retrieval name: RZTKInformation Retrieval dataset: name: rusisms ru title type: rusisms-ru-title metrics: - type: dot_accuracy_1 value: 0.8384615384615385 name: Dot Accuracy 1 - type: dot_accuracy_3 value: 0.8923076923076924 name: Dot Accuracy 3 - type: dot_accuracy_5 value: 0.9076923076923077 name: Dot Accuracy 5 - type: dot_accuracy_10 value: 0.9307692307692308 name: Dot Accuracy 10 - type: dot_precision_1 value: 0.8384615384615385 name: Dot Precision 1 - type: dot_precision_3 value: 0.7794871794871796 name: Dot Precision 3 - type: dot_precision_5 value: 0.7338461538461538 name: Dot Precision 5 - type: dot_precision_10 value: 0.6276923076923077 name: Dot Precision 10 - type: dot_recall_1 value: 0.16806442070486566 name: Dot Recall 1 - type: dot_recall_3 value: 0.34858717229798075 name: Dot Recall 3 - type: dot_recall_5 value: 0.48036976527090636 name: Dot Recall 5 - type: dot_recall_10 value: 0.6880841059010939 name: Dot Recall 10 - type: dot_ndcg_10 value: 0.8371088333291086 name: Dot Ndcg 10 - type: dot_mrr_10 value: 0.8705219780219781 name: Dot Mrr 10 - type: dot_map_100 value: 0.834520738484658 name: Dot Map 100 - task: type: rztkinformation-retrieval name: RZTKInformation Retrieval dataset: name: rusisms uk options type: rusisms-uk-options metrics: - type: dot_accuracy_1 value: 0.676923076923077 name: Dot Accuracy 1 - type: dot_accuracy_3 value: 0.7615384615384615 name: Dot Accuracy 3 - type: dot_accuracy_5 value: 0.8153846153846154 name: Dot Accuracy 5 - type: dot_accuracy_10 value: 0.8923076923076924 name: Dot Accuracy 10 - type: dot_precision_1 value: 0.676923076923077 name: Dot Precision 1 - type: dot_precision_3 value: 0.6435897435897436 name: Dot Precision 3 - type: dot_precision_5 value: 0.6184615384615385 name: Dot Precision 5 - type: dot_precision_10 value: 0.5530769230769231 name: Dot Precision 10 - type: dot_recall_1 value: 0.13312045127560207 name: Dot Recall 1 - type: dot_recall_3 value: 0.30128290449835693 name: Dot Recall 3 - type: dot_recall_5 value: 0.4195731612041784 name: Dot Recall 5 - type: dot_recall_10 value: 0.6054121983452854 name: Dot Recall 10 - type: dot_ndcg_10 value: 0.7227342790210933 name: Dot Ndcg 10 - type: dot_mrr_10 value: 0.7384340659340659 name: Dot Mrr 10 - type: dot_map_100 value: 0.7288085980763663 name: Dot Map 100 - task: type: rztkinformation-retrieval name: RZTKInformation Retrieval dataset: name: rusisms ru options type: rusisms-ru-options metrics: - type: dot_accuracy_1 value: 0.7307692307692307 name: Dot Accuracy 1 - type: dot_accuracy_3 value: 0.8153846153846154 name: Dot Accuracy 3 - type: dot_accuracy_5 value: 0.8769230769230769 name: Dot Accuracy 5 - type: dot_accuracy_10 value: 0.8923076923076924 name: Dot Accuracy 10 - type: dot_precision_1 value: 0.7307692307692307 name: Dot Precision 1 - type: dot_precision_3 value: 0.6692307692307692 name: Dot Precision 3 - type: dot_precision_5 value: 0.6399999999999999 name: Dot Precision 5 - type: dot_precision_10 value: 0.563076923076923 name: Dot Precision 10 - type: dot_recall_1 value: 0.14490921806436888 name: Dot Recall 1 - type: dot_recall_3 value: 0.30560677507222755 name: Dot Recall 3 - type: dot_recall_5 value: 0.4247145475955648 name: Dot Recall 5 - type: dot_recall_10 value: 0.6205841386473162 name: Dot Recall 10 - type: dot_ndcg_10 value: 0.744382870641256 name: Dot Ndcg 10 - type: dot_mrr_10 value: 0.7852777777777777 name: Dot Mrr 10 - type: dot_map_100 value: 0.7399821840526332 name: Dot Map 100 - task: type: rztkinformation-retrieval name: RZTKInformation Retrieval dataset: name: rusisms corrected uk title type: rusisms_corrected-uk-title metrics: - type: dot_accuracy_1 value: 0.9230769230769231 name: Dot Accuracy 1 - type: dot_accuracy_3 value: 0.9769230769230769 name: Dot Accuracy 3 - type: dot_accuracy_5 value: 0.9769230769230769 name: Dot Accuracy 5 - type: dot_accuracy_10 value: 1.0 name: Dot Accuracy 10 - type: dot_precision_1 value: 0.9230769230769231 name: Dot Precision 1 - type: dot_precision_3 value: 0.841025641025641 name: Dot Precision 3 - type: dot_precision_5 value: 0.7907692307692308 name: Dot Precision 5 - type: dot_precision_10 value: 0.6753846153846154 name: Dot Precision 10 - type: dot_recall_1 value: 0.19258995555648747 name: Dot Recall 1 - type: dot_recall_3 value: 0.3984948069338763 name: Dot Recall 3 - type: dot_recall_5 value: 0.5337760907024034 name: Dot Recall 5 - type: dot_recall_10 value: 0.7618210364747455 name: Dot Recall 10 - type: dot_ndcg_10 value: 0.9170248499517376 name: Dot Ndcg 10 - type: dot_mrr_10 value: 0.9516239316239314 name: Dot Mrr 10 - type: dot_map_100 value: 0.9045707174129692 name: Dot Map 100 - task: type: rztkinformation-retrieval name: RZTKInformation Retrieval dataset: name: rusisms corrected ru title type: rusisms_corrected-ru-title metrics: - type: dot_accuracy_1 value: 0.9 name: Dot Accuracy 1 - type: dot_accuracy_3 value: 0.9615384615384616 name: Dot Accuracy 3 - type: dot_accuracy_5 value: 0.9769230769230769 name: Dot Accuracy 5 - type: dot_accuracy_10 value: 1.0 name: Dot Accuracy 10 - type: dot_precision_1 value: 0.9 name: Dot Precision 1 - type: dot_precision_3 value: 0.8358974358974358 name: Dot Precision 3 - type: dot_precision_5 value: 0.7784615384615385 name: Dot Precision 5 - type: dot_precision_10 value: 0.6661538461538461 name: Dot Precision 10 - type: dot_recall_1 value: 0.18966260512913705 name: Dot Recall 1 - type: dot_recall_3 value: 0.3888551410442104 name: Dot Recall 3 - type: dot_recall_5 value: 0.5267335645151403 name: Dot Recall 5 - type: dot_recall_10 value: 0.748230599134308 name: Dot Recall 10 - type: dot_ndcg_10 value: 0.9014737854592502 name: Dot Ndcg 10 - type: dot_mrr_10 value: 0.933482905982906 name: Dot Mrr 10 - type: dot_map_100 value: 0.8925893888838562 name: Dot Map 100 - task: type: rztkinformation-retrieval name: RZTKInformation Retrieval dataset: name: rusisms corrected uk options type: rusisms_corrected-uk-options metrics: - type: dot_accuracy_1 value: 0.7615384615384615 name: Dot Accuracy 1 - type: dot_accuracy_3 value: 0.9 name: Dot Accuracy 3 - type: dot_accuracy_5 value: 0.9307692307692308 name: Dot Accuracy 5 - type: dot_accuracy_10 value: 0.9846153846153847 name: Dot Accuracy 10 - type: dot_precision_1 value: 0.7615384615384615 name: Dot Precision 1 - type: dot_precision_3 value: 0.735897435897436 name: Dot Precision 3 - type: dot_precision_5 value: 0.7076923076923077 name: Dot Precision 5 - type: dot_precision_10 value: 0.6415384615384615 name: Dot Precision 10 - type: dot_recall_1 value: 0.15392219529816992 name: Dot Recall 1 - type: dot_recall_3 value: 0.35000920112241907 name: Dot Recall 3 - type: dot_recall_5 value: 0.4703239382796628 name: Dot Recall 5 - type: dot_recall_10 value: 0.7181653977378374 name: Dot Recall 10 - type: dot_ndcg_10 value: 0.8357927469132899 name: Dot Ndcg 10 - type: dot_mrr_10 value: 0.8372954822954823 name: Dot Mrr 10 - type: dot_map_100 value: 0.828044801612513 name: Dot Map 100 - task: type: rztkinformation-retrieval name: RZTKInformation Retrieval dataset: name: rusisms corrected ru options type: rusisms_corrected-ru-options metrics: - type: dot_accuracy_1 value: 0.8153846153846154 name: Dot Accuracy 1 - type: dot_accuracy_3 value: 0.9307692307692308 name: Dot Accuracy 3 - type: dot_accuracy_5 value: 0.9615384615384616 name: Dot Accuracy 5 - type: dot_accuracy_10 value: 0.9769230769230769 name: Dot Accuracy 10 - type: dot_precision_1 value: 0.8153846153846154 name: Dot Precision 1 - type: dot_precision_3 value: 0.7692307692307693 name: Dot Precision 3 - type: dot_precision_5 value: 0.7307692307692307 name: Dot Precision 5 - type: dot_precision_10 value: 0.6461538461538461 name: Dot Precision 10 - type: dot_recall_1 value: 0.16098154934281808 name: Dot Recall 1 - type: dot_recall_3 value: 0.3597065118955812 name: Dot Recall 3 - type: dot_recall_5 value: 0.5030974929843319 name: Dot Recall 5 - type: dot_recall_10 value: 0.7224944755227974 name: Dot Recall 10 - type: dot_ndcg_10 value: 0.8566150552642338 name: Dot Ndcg 10 - type: dot_mrr_10 value: 0.8796336996336996 name: Dot Mrr 10 - type: dot_map_100 value: 0.8437060147318505 name: Dot Map 100 - task: type: rztkinformation-retrieval name: RZTKInformation Retrieval dataset: name: core typos uk title type: core_typos-uk-title metrics: - type: dot_accuracy_1 value: 0.6902887139107612 name: Dot Accuracy 1 - type: dot_accuracy_3 value: 0.8503937007874016 name: Dot Accuracy 3 - type: dot_accuracy_5 value: 0.9094488188976378 name: Dot Accuracy 5 - type: dot_accuracy_10 value: 0.9606299212598425 name: Dot Accuracy 10 - type: dot_precision_1 value: 0.6902887139107612 name: Dot Precision 1 - type: dot_precision_3 value: 0.6360454943132109 name: Dot Precision 3 - type: dot_precision_5 value: 0.5593175853018373 name: Dot Precision 5 - type: dot_precision_10 value: 0.3586614173228347 name: Dot Precision 10 - type: dot_recall_1 value: 0.20047962754655668 name: Dot Recall 1 - type: dot_recall_3 value: 0.4984340499104279 name: Dot Recall 3 - type: dot_recall_5 value: 0.69014602341374 name: Dot Recall 5 - type: dot_recall_10 value: 0.8553529767112444 name: Dot Recall 10 - type: dot_ndcg_10 value: 0.7686270697676233 name: Dot Ndcg 10 - type: dot_mrr_10 value: 0.7825412448443944 name: Dot Mrr 10 - type: dot_map_100 value: 0.7184809513160368 name: Dot Map 100 - task: type: rztkinformation-retrieval name: RZTKInformation Retrieval dataset: name: core typos ru title type: core_typos-ru-title metrics: - type: dot_accuracy_1 value: 0.6889763779527559 name: Dot Accuracy 1 - type: dot_accuracy_3 value: 0.8530183727034121 name: Dot Accuracy 3 - type: dot_accuracy_5 value: 0.9120734908136483 name: Dot Accuracy 5 - type: dot_accuracy_10 value: 0.9619422572178478 name: Dot Accuracy 10 - type: dot_precision_1 value: 0.6889763779527559 name: Dot Precision 1 - type: dot_precision_3 value: 0.6377952755905512 name: Dot Precision 3 - type: dot_precision_5 value: 0.5608923884514435 name: Dot Precision 5 - type: dot_precision_10 value: 0.35879265091863516 name: Dot Precision 10 - type: dot_recall_1 value: 0.2019065325167687 name: Dot Recall 1 - type: dot_recall_3 value: 0.5041270882806315 name: Dot Recall 3 - type: dot_recall_5 value: 0.6928071491063618 name: Dot Recall 5 - type: dot_recall_10 value: 0.8585536182977128 name: Dot Recall 10 - type: dot_ndcg_10 value: 0.7699411890838681 name: Dot Ndcg 10 - type: dot_mrr_10 value: 0.7819496521268174 name: Dot Mrr 10 - type: dot_map_100 value: 0.7190649802215707 name: Dot Map 100 - task: type: rztkinformation-retrieval name: RZTKInformation Retrieval dataset: name: core typos uk options type: core_typos-uk-options metrics: - type: dot_accuracy_1 value: 0.5774278215223098 name: Dot Accuracy 1 - type: dot_accuracy_3 value: 0.7427821522309711 name: Dot Accuracy 3 - type: dot_accuracy_5 value: 0.8097112860892388 name: Dot Accuracy 5 - type: dot_accuracy_10 value: 0.8727034120734908 name: Dot Accuracy 10 - type: dot_precision_1 value: 0.5774278215223098 name: Dot Precision 1 - type: dot_precision_3 value: 0.5201224846894139 name: Dot Precision 3 - type: dot_precision_5 value: 0.46430446194225733 name: Dot Precision 5 - type: dot_precision_10 value: 0.3148293963254593 name: Dot Precision 10 - type: dot_recall_1 value: 0.1598211681873099 name: Dot Recall 1 - type: dot_recall_3 value: 0.39694204891055285 name: Dot Recall 3 - type: dot_recall_5 value: 0.5620844269466317 name: Dot Recall 5 - type: dot_recall_10 value: 0.7416458359371746 name: Dot Recall 10 - type: dot_ndcg_10 value: 0.6492253136169239 name: Dot Ndcg 10 - type: dot_mrr_10 value: 0.6747786734991458 name: Dot Mrr 10 - type: dot_map_100 value: 0.5991438141662567 name: Dot Map 100 - task: type: rztkinformation-retrieval name: RZTKInformation Retrieval dataset: name: core typos ru options type: core_typos-ru-options metrics: - type: dot_accuracy_1 value: 0.5761154855643045 name: Dot Accuracy 1 - type: dot_accuracy_3 value: 0.7401574803149606 name: Dot Accuracy 3 - type: dot_accuracy_5 value: 0.8057742782152231 name: Dot Accuracy 5 - type: dot_accuracy_10 value: 0.8779527559055118 name: Dot Accuracy 10 - type: dot_precision_1 value: 0.5761154855643045 name: Dot Precision 1 - type: dot_precision_3 value: 0.5227471566054243 name: Dot Precision 3 - type: dot_precision_5 value: 0.4674540682414698 name: Dot Precision 5 - type: dot_precision_10 value: 0.3139107611548556 name: Dot Precision 10 - type: dot_recall_1 value: 0.1653933883264592 name: Dot Recall 1 - type: dot_recall_3 value: 0.400884785235179 name: Dot Recall 3 - type: dot_recall_5 value: 0.5683497896096321 name: Dot Recall 5 - type: dot_recall_10 value: 0.7359840436612088 name: Dot Recall 10 - type: dot_ndcg_10 value: 0.650193326341318 name: Dot Ndcg 10 - type: dot_mrr_10 value: 0.6749359455068115 name: Dot Mrr 10 - type: dot_map_100 value: 0.6039222286208336 name: Dot Map 100 - task: type: rztkinformation-retrieval name: RZTKInformation Retrieval dataset: name: 'bm full matryoshka dim 768 ' type: bm-full--matryoshka_dim-768-- metrics: - type: dot_accuracy_1 value: 0.6325136612021858 name: Dot Accuracy 1 - type: dot_precision_1 value: 0.6325136612021858 name: Dot Precision 1 - type: dot_recall_1 value: 0.04407246382453946 name: Dot Recall 1 - type: dot_ndcg_1 value: 0.6325136612021858 name: Dot Ndcg 1 - type: dot_mrr_1 value: 0.6325136612021858 name: Dot Mrr 1 - type: dot_map_100 value: 0.5595201386694402 name: Dot Map 100 - task: type: rztkinformation-retrieval name: RZTKInformation Retrieval dataset: name: 'bm full matryoshka dim 512 ' type: bm-full--matryoshka_dim-512-- metrics: - type: dot_accuracy_1 value: 0.6193078324225865 name: Dot Accuracy 1 - type: dot_precision_1 value: 0.6193078324225865 name: Dot Precision 1 - type: dot_recall_1 value: 0.04369236548131633 name: Dot Recall 1 - type: dot_ndcg_1 value: 0.6193078324225865 name: Dot Ndcg 1 - type: dot_mrr_1 value: 0.6193078324225865 name: Dot Mrr 1 - type: dot_map_100 value: 0.5515211390810298 name: Dot Map 100 - task: type: rztkinformation-retrieval name: RZTKInformation Retrieval dataset: name: 'bm full matryoshka dim 256 ' type: bm-full--matryoshka_dim-256-- metrics: - type: dot_accuracy_1 value: 0.6138433515482696 name: Dot Accuracy 1 - type: dot_precision_1 value: 0.6138433515482696 name: Dot Precision 1 - type: dot_recall_1 value: 0.04255542742758193 name: Dot Recall 1 - type: dot_ndcg_1 value: 0.6138433515482696 name: Dot Ndcg 1 - type: dot_mrr_1 value: 0.6138433515482696 name: Dot Mrr 1 - type: dot_map_100 value: 0.5385524531355951 name: Dot Map 100 - task: type: rztkinformation-retrieval name: RZTKInformation Retrieval dataset: name: 'bm full matryoshka dim 128 ' type: bm-full--matryoshka_dim-128-- metrics: - type: dot_accuracy_1 value: 0.5915300546448088 name: Dot Accuracy 1 - type: dot_precision_1 value: 0.5915300546448088 name: Dot Precision 1 - type: dot_recall_1 value: 0.04021446721014544 name: Dot Recall 1 - type: dot_ndcg_1 value: 0.5915300546448088 name: Dot Ndcg 1 - type: dot_mrr_1 value: 0.5915300546448088 name: Dot Mrr 1 - type: dot_map_100 value: 0.5056118434496266 name: Dot Map 100 --- # SentenceTransformer based on intfloat/multilingual-e5-base This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [intfloat/multilingual-e5-base](https://huggingface.co./intfloat/multilingual-e5-base) on the rozetka_positive_pairs dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. ## Model Details ### Model Description - **Model Type:** Sentence Transformer - **Base model:** [intfloat/multilingual-e5-base](https://huggingface.co./intfloat/multilingual-e5-base) - **Maximum Sequence Length:** 512 tokens - **Output Dimensionality:** 768 dimensions - **Similarity Function:** Dot Product - **Training Dataset:** - rozetka_positive_pairs ### Model Sources - **Documentation:** [Sentence Transformers Documentation](https://sbert.net) - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers) - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co./models?library=sentence-transformers) ### Full Model Architecture ``` RZTKSentenceTransformer( (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: XLMRobertaModel (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) (2): Normalize() ) ``` ## Usage ### Direct Usage (Sentence Transformers) First install the Sentence Transformers library: ```bash pip install -U sentence-transformers ``` Then you can load this model and run inference. ```python from sentence_transformers import SentenceTransformer # Download from the 🤗 Hub model = SentenceTransformer("yklymchuk-rztk/multilingual-e5-base-matryoshka2d-mnr-12") # Run inference sentences = [ 'query: джинсовые шорты женские', 'passage: Шорты джинсовые женские H&M 0612481-01 36 Темно-серые (KAY2000000788180)', 'passage: Ремешок MilaneseBand для Apple Watch 49 | 45 | 44 | 42 mm Black (UP46001)', ] embeddings = model.encode(sentences) print(embeddings.shape) # [3, 768] # Get the similarity scores for the embeddings similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] ``` ## Evaluation ### Metrics #### RZTKInformation Retrieval * Dataset: `validation--matryoshka_dim-768--` * Evaluated with sentence_transformers_training.evaluation.information_retrieval_evaluator.RZTKInformationRetrievalEvaluator | Metric | Value | |:-----------------|:-----------| | dot_accuracy_10 | 0.4731 | | dot_precision_10 | 0.0602 | | dot_recall_10 | 0.4249 | | **dot_ndcg_10** | **0.2692** | | dot_mrr_10 | 0.2314 | | dot_map_60 | 0.2289 | #### RZTKInformation Retrieval * Datasets: `bm-full`, `core-uk-title`, `core-ru-title`, `core-uk-options`, `core-ru-options`, `options-uk-title`, `options-ru-title`, `options-uk-options`, `options-ru-options`, `rusisms-uk-title`, `rusisms-ru-title`, `rusisms-uk-options`, `rusisms-ru-options`, `rusisms_corrected-uk-title`, `rusisms_corrected-ru-title`, `rusisms_corrected-uk-options`, `rusisms_corrected-ru-options`, `core_typos-uk-title`, `core_typos-ru-title`, `core_typos-uk-options` and `core_typos-ru-options` * Evaluated with sentence_transformers_training.evaluation.information_retrieval_evaluator.RZTKInformationRetrievalEvaluator | Metric | bm-full | core-uk-title | core-ru-title | core-uk-options | core-ru-options | options-uk-title | options-ru-title | options-uk-options | options-ru-options | rusisms-uk-title | rusisms-ru-title | rusisms-uk-options | rusisms-ru-options | rusisms_corrected-uk-title | rusisms_corrected-ru-title | rusisms_corrected-uk-options | rusisms_corrected-ru-options | core_typos-uk-title | core_typos-ru-title | core_typos-uk-options | core_typos-ru-options | |:-----------------|:----------|:--------------|:--------------|:----------------|:----------------|:-----------------|:-----------------|:-------------------|:-------------------|:-----------------|:-----------------|:-------------------|:-------------------|:---------------------------|:---------------------------|:-----------------------------|:-----------------------------|:--------------------|:--------------------|:----------------------|:----------------------| | dot_accuracy_1 | 0.6325 | 0.7874 | 0.7887 | 0.6667 | 0.664 | 0.8228 | 0.8277 | 0.6966 | 0.6966 | 0.8308 | 0.8385 | 0.6769 | 0.7308 | 0.9231 | 0.9 | 0.7615 | 0.8154 | 0.6903 | 0.689 | 0.5774 | 0.5761 | | dot_accuracy_3 | 0.7454 | 0.9239 | 0.9239 | 0.8386 | 0.8425 | 0.9369 | 0.9466 | 0.8568 | 0.8641 | 0.8846 | 0.8923 | 0.7615 | 0.8154 | 0.9769 | 0.9615 | 0.9 | 0.9308 | 0.8504 | 0.853 | 0.7428 | 0.7402 | | dot_accuracy_5 | 0.801 | 0.9633 | 0.9698 | 0.8976 | 0.9016 | 0.9733 | 0.9757 | 0.9126 | 0.9199 | 0.9154 | 0.9077 | 0.8154 | 0.8769 | 0.9769 | 0.9769 | 0.9308 | 0.9615 | 0.9094 | 0.9121 | 0.8097 | 0.8058 | | dot_accuracy_10 | 0.8657 | 0.9908 | 0.9908 | 0.9593 | 0.9567 | 0.9927 | 0.9927 | 0.966 | 0.9563 | 0.9385 | 0.9308 | 0.8923 | 0.8923 | 1.0 | 1.0 | 0.9846 | 0.9769 | 0.9606 | 0.9619 | 0.8727 | 0.878 | | dot_precision_1 | 0.6325 | 0.7874 | 0.7887 | 0.6667 | 0.664 | 0.8228 | 0.8277 | 0.6966 | 0.6966 | 0.8308 | 0.8385 | 0.6769 | 0.7308 | 0.9231 | 0.9 | 0.7615 | 0.8154 | 0.6903 | 0.689 | 0.5774 | 0.5761 | | dot_precision_3 | 0.6213 | 0.7108 | 0.7161 | 0.6067 | 0.6115 | 0.7379 | 0.7443 | 0.6359 | 0.623 | 0.7821 | 0.7795 | 0.6436 | 0.6692 | 0.841 | 0.8359 | 0.7359 | 0.7692 | 0.636 | 0.6378 | 0.5201 | 0.5227 | | dot_precision_5 | 0.606 | 0.6252 | 0.6278 | 0.5488 | 0.5493 | 0.5869 | 0.5859 | 0.5068 | 0.5078 | 0.7354 | 0.7338 | 0.6185 | 0.64 | 0.7908 | 0.7785 | 0.7077 | 0.7308 | 0.5593 | 0.5609 | 0.4643 | 0.4675 | | dot_precision_10 | 0.562 | 0.3891 | 0.3882 | 0.3635 | 0.3593 | 0.3362 | 0.3367 | 0.3083 | 0.3066 | 0.6315 | 0.6277 | 0.5531 | 0.5631 | 0.6754 | 0.6662 | 0.6415 | 0.6462 | 0.3587 | 0.3588 | 0.3148 | 0.3139 | | dot_recall_1 | 0.0441 | 0.241 | 0.242 | 0.1948 | 0.197 | 0.2578 | 0.2582 | 0.2136 | 0.2145 | 0.1592 | 0.1681 | 0.1331 | 0.1449 | 0.1926 | 0.1897 | 0.1539 | 0.161 | 0.2005 | 0.2019 | 0.1598 | 0.1654 | | dot_recall_3 | 0.1264 | 0.5665 | 0.5687 | 0.4692 | 0.4768 | 0.66 | 0.6647 | 0.5664 | 0.5557 | 0.3476 | 0.3486 | 0.3013 | 0.3056 | 0.3985 | 0.3889 | 0.35 | 0.3597 | 0.4984 | 0.5041 | 0.3969 | 0.4009 | | dot_recall_5 | 0.1989 | 0.7759 | 0.7834 | 0.6658 | 0.6704 | 0.8477 | 0.8462 | 0.7313 | 0.7298 | 0.4838 | 0.4804 | 0.4196 | 0.4247 | 0.5338 | 0.5267 | 0.4703 | 0.5031 | 0.6901 | 0.6928 | 0.5621 | 0.5683 | | dot_recall_10 | 0.3427 | 0.9339 | 0.932 | 0.8585 | 0.8482 | 0.96 | 0.9612 | 0.8752 | 0.87 | 0.6879 | 0.6881 | 0.6054 | 0.6206 | 0.7618 | 0.7482 | 0.7182 | 0.7225 | 0.8554 | 0.8586 | 0.7416 | 0.736 | | **dot_ndcg_10** | **0.602** | **0.856** | **0.8555** | **0.759** | **0.7556** | **0.8799** | **0.8823** | **0.7777** | **0.773** | **0.8363** | **0.8371** | **0.7227** | **0.7444** | **0.917** | **0.9015** | **0.8358** | **0.8566** | **0.7686** | **0.7699** | **0.6492** | **0.6502** | | dot_mrr_10 | 0.6988 | 0.8623 | 0.8609 | 0.7659 | 0.7664 | 0.8843 | 0.8893 | 0.7871 | 0.7869 | 0.8671 | 0.8705 | 0.7384 | 0.7853 | 0.9516 | 0.9335 | 0.8373 | 0.8796 | 0.7825 | 0.7819 | 0.6748 | 0.6749 | | dot_map_100 | 0.5595 | 0.8088 | 0.8101 | 0.7042 | 0.7052 | 0.8247 | 0.8268 | 0.7193 | 0.713 | 0.8295 | 0.8345 | 0.7288 | 0.74 | 0.9046 | 0.8926 | 0.828 | 0.8437 | 0.7185 | 0.7191 | 0.5991 | 0.6039 | #### RZTKInformation Retrieval * Datasets: `bm-full--matryoshka_dim-768--`, `bm-full--matryoshka_dim-512--`, `bm-full--matryoshka_dim-256--` and `bm-full--matryoshka_dim-128--` * Evaluated with sentence_transformers_training.evaluation.information_retrieval_evaluator.RZTKInformationRetrievalEvaluator | Metric | bm-full--matryoshka_dim-768-- | bm-full--matryoshka_dim-512-- | bm-full--matryoshka_dim-256-- | bm-full--matryoshka_dim-128-- | |:----------------|:------------------------------|:------------------------------|:------------------------------|:------------------------------| | dot_accuracy_1 | 0.6325 | 0.6193 | 0.6138 | 0.5915 | | dot_precision_1 | 0.6325 | 0.6193 | 0.6138 | 0.5915 | | dot_recall_1 | 0.0441 | 0.0437 | 0.0426 | 0.0402 | | **dot_ndcg_1** | **0.6325** | **0.6193** | **0.6138** | **0.5915** | | dot_mrr_1 | 0.6325 | 0.6193 | 0.6138 | 0.5915 | | dot_map_100 | 0.5595 | 0.5515 | 0.5386 | 0.5056 | ## Training Details ### Training Dataset #### rozetka_positive_pairs * Dataset: rozetka_positive_pairs * Size: 41,479,299 training samples * Columns: query and text * Approximate statistics based on the first 1000 samples: | | query | text | |:--------|:----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------| | type | string | string | | details | | | * Samples: | query | text | |:----------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | query: campingaz fold n cool classic 10l dark blue | passage: Термосумка Campingaz Fold'n Cool Classic 10L Dark Blue (4823082704729) | | query: campingaz fold n cool classic 10l dark blue | passage: Термопродукція Campingaz Гарантія 14 днів Вид Термосумки Колір Синій з білим Режим роботи Охолодження Країна реєстрації бренда Франція Країна-виробник товару Китай Тип гарантійного талона Гарантія по чеку Можливість доставки Почтомати Доставка Premium Немає | | query: campingaz fold n cool classic 10l dark blue | passage: Термосумка Campingaz Fold'n Cool Classic 10L Dark Blue (4823082704729) | * Loss: sentence_transformers_training.model.matryoshka2d_loss.RZTKMatryoshka2dLoss with these parameters: ```json { "loss": "RZTKMultipleNegativesRankingLoss", "n_layers_per_step": 1, "last_layer_weight": 1.0, "prior_layers_weight": 1.0, "kl_div_weight": 1.0, "kl_temperature": 0.3, "matryoshka_dims": [ 768, 512, 256, 128 ], "matryoshka_weights": [ 1, 1, 1, 1 ], "n_dims_per_step": 1 } ``` ### Evaluation Dataset #### rozetka_positive_pairs * Dataset: rozetka_positive_pairs * Size: 618,868 evaluation samples * Columns: query and text * Approximate statistics based on the first 1000 samples: | | query | text | |:--------|:----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------| | type | string | string | | details | | | * Samples: | query | text | |:-----------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | query: casio dw | passage: Наручний чоловічий годинник Casio DW-5600BB-1ER | | query: casio dw | passage: Наручні годинники Casio Гарантія 24 місяці Механізм Кварцовий Тип Чоловічі Форма корпусу Прямокутна Матеріал корпусу Полімер Скло Мінеральне Водонепроникність 200 м Матеріал ремінця/браслета Полімер Тип циферблата Електронний | | query: casio dw | passage: Наручные мужские часы Casio DW-5600BB-1ER | * Loss: sentence_transformers_training.model.matryoshka2d_loss.RZTKMatryoshka2dLoss with these parameters: ```json { "loss": "RZTKMultipleNegativesRankingLoss", "n_layers_per_step": 1, "last_layer_weight": 1.0, "prior_layers_weight": 1.0, "kl_div_weight": 1.0, "kl_temperature": 0.3, "matryoshka_dims": [ 768, 512, 256, 128 ], "matryoshka_weights": [ 1, 1, 1, 1 ], "n_dims_per_step": 1 } ``` ### Training Hyperparameters #### Non-Default Hyperparameters - `eval_strategy`: steps - `per_device_train_batch_size`: 88 - `per_device_eval_batch_size`: 88 - `learning_rate`: 2e-05 - `num_train_epochs`: 1.0 - `warmup_ratio`: 0.1 - `bf16`: True - `bf16_full_eval`: True - `tf32`: True - `dataloader_num_workers`: 4 - `load_best_model_at_end`: True - `optim`: adafactor - `push_to_hub`: True - `hub_model_id`: yklymchuk-rztk/multilingual-e5-base-matryoshka2d-mnr-12 - `hub_private_repo`: True - `prompts`: {'query': 'query: ', 'text': 'passage: '} - `batch_sampler`: no_duplicates #### All Hyperparameters
Click to expand - `overwrite_output_dir`: False - `do_predict`: False - `eval_strategy`: steps - `prediction_loss_only`: True - `per_device_train_batch_size`: 88 - `per_device_eval_batch_size`: 88 - `per_gpu_train_batch_size`: None - `per_gpu_eval_batch_size`: None - `gradient_accumulation_steps`: 1 - `eval_accumulation_steps`: None - `torch_empty_cache_steps`: None - `learning_rate`: 2e-05 - `weight_decay`: 0.0 - `adam_beta1`: 0.9 - `adam_beta2`: 0.999 - `adam_epsilon`: 1e-08 - `max_grad_norm`: 1.0 - `num_train_epochs`: 1.0 - `max_steps`: -1 - `lr_scheduler_type`: linear - `lr_scheduler_kwargs`: {} - `warmup_ratio`: 0.1 - `warmup_steps`: 0 - `log_level`: passive - `log_level_replica`: warning - `log_on_each_node`: True - `logging_nan_inf_filter`: True - `save_safetensors`: True - `save_on_each_node`: False - `save_only_model`: False - `restore_callback_states_from_checkpoint`: False - `no_cuda`: False - `use_cpu`: False - `use_mps_device`: False - `seed`: 42 - `data_seed`: None - `jit_mode_eval`: False - `use_ipex`: False - `bf16`: True - `fp16`: False - `fp16_opt_level`: O1 - `half_precision_backend`: auto - `bf16_full_eval`: True - `fp16_full_eval`: False - `tf32`: True - `local_rank`: 0 - `ddp_backend`: None - `tpu_num_cores`: None - `tpu_metrics_debug`: False - `debug`: [] - `dataloader_drop_last`: True - `dataloader_num_workers`: 4 - `dataloader_prefetch_factor`: None - `past_index`: -1 - `disable_tqdm`: False - `remove_unused_columns`: True - `label_names`: None - `load_best_model_at_end`: True - `ignore_data_skip`: False - `fsdp`: [] - `fsdp_min_num_params`: 0 - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} - `fsdp_transformer_layer_cls_to_wrap`: None - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} - `deepspeed`: None - `label_smoothing_factor`: 0.0 - `optim`: adafactor - `optim_args`: None - `adafactor`: False - `group_by_length`: False - `length_column_name`: length - `ddp_find_unused_parameters`: None - `ddp_bucket_cap_mb`: None - `ddp_broadcast_buffers`: False - `dataloader_pin_memory`: True - `dataloader_persistent_workers`: False - `skip_memory_metrics`: True - `use_legacy_prediction_loop`: False - `push_to_hub`: True - `resume_from_checkpoint`: None - `hub_model_id`: yklymchuk-rztk/multilingual-e5-base-matryoshka2d-mnr-12 - `hub_strategy`: every_save - `hub_private_repo`: True - `hub_always_push`: False - `gradient_checkpointing`: False - `gradient_checkpointing_kwargs`: None - `include_inputs_for_metrics`: False - `include_for_metrics`: [] - `eval_do_concat_batches`: True - `fp16_backend`: auto - `push_to_hub_model_id`: None - `push_to_hub_organization`: None - `mp_parameters`: - `auto_find_batch_size`: False - `full_determinism`: False - `torchdynamo`: None - `ray_scope`: last - `ddp_timeout`: 1800 - `torch_compile`: False - `torch_compile_backend`: None - `torch_compile_mode`: None - `dispatch_batches`: None - `split_batches`: None - `include_tokens_per_second`: False - `include_num_input_tokens_seen`: False - `neftune_noise_alpha`: None - `optim_target_modules`: None - `batch_eval_metrics`: False - `eval_on_start`: False - `use_liger_kernel`: False - `eval_use_gather_object`: False - `average_tokens_across_devices`: False - `prompts`: {'query': 'query: ', 'text': 'passage: '} - `batch_sampler`: no_duplicates - `multi_dataset_batch_sampler`: proportional - `ddp_static_graph`: False - `ddp_comm_hook`: bf16 - `gradient_as_bucket_view`: False - `num_proc`: 30
### Training Logs
Click to expand | Epoch | Step | Training Loss | Validation Loss | validation--matryoshka_dim-768--_dot_ndcg_10 | bm-full_dot_ndcg_10 | core-uk-title_dot_ndcg_10 | core-ru-title_dot_ndcg_10 | core-uk-options_dot_ndcg_10 | core-ru-options_dot_ndcg_10 | options-uk-title_dot_ndcg_10 | options-ru-title_dot_ndcg_10 | options-uk-options_dot_ndcg_10 | options-ru-options_dot_ndcg_10 | rusisms-uk-title_dot_ndcg_10 | rusisms-ru-title_dot_ndcg_10 | rusisms-uk-options_dot_ndcg_10 | rusisms-ru-options_dot_ndcg_10 | rusisms_corrected-uk-title_dot_ndcg_10 | rusisms_corrected-ru-title_dot_ndcg_10 | rusisms_corrected-uk-options_dot_ndcg_10 | rusisms_corrected-ru-options_dot_ndcg_10 | core_typos-uk-title_dot_ndcg_10 | core_typos-ru-title_dot_ndcg_10 | core_typos-uk-options_dot_ndcg_10 | core_typos-ru-options_dot_ndcg_10 | bm-full--matryoshka_dim-768--_dot_ndcg_1 | bm-full--matryoshka_dim-512--_dot_ndcg_1 | bm-full--matryoshka_dim-256--_dot_ndcg_1 | bm-full--matryoshka_dim-128--_dot_ndcg_1 | |:-------:|:---------:|:-------------:|:---------------:|:--------------------------------------------:|:-------------------:|:-------------------------:|:-------------------------:|:---------------------------:|:---------------------------:|:----------------------------:|:----------------------------:|:------------------------------:|:------------------------------:|:----------------------------:|:----------------------------:|:------------------------------:|:------------------------------:|:--------------------------------------:|:--------------------------------------:|:----------------------------------------:|:----------------------------------------:|:-------------------------------:|:-------------------------------:|:---------------------------------:|:---------------------------------:|:----------------------------------------:|:----------------------------------------:|:----------------------------------------:|:----------------------------------------:| | 0.0050 | 590 | 4.4654 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.0100 | 1180 | 4.3301 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.0150 | 1770 | 3.8526 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.0200 | 2360 | 3.0371 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.0250 | 2950 | 2.3669 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.0300 | 3540 | 1.917 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.0350 | 4130 | 1.7046 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.0401 | 4720 | 1.5069 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.0451 | 5310 | 1.5094 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.0501 | 5900 | 1.3703 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.0551 | 6490 | 1.356 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.0601 | 7080 | 1.2088 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.0651 | 7670 | 1.2458 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.0701 | 8260 | 1.1176 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.0751 | 8850 | 1.0877 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.0801 | 9440 | 1.0321 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.0851 | 10030 | 0.9977 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.0901 | 10620 | 0.9506 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.0951 | 11210 | 0.9256 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.1000 | 11784 | - | 0.7840 | 0.2497 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.1001 | 11800 | 0.9233 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.1051 | 12390 | 0.8985 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.1102 | 12980 | 0.8473 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.1152 | 13570 | 0.7937 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.1202 | 14160 | 0.7781 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.1252 | 14750 | 0.7729 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.1302 | 15340 | 0.7478 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.1352 | 15930 | 0.7318 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.1402 | 16520 | 0.7201 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.1452 | 17110 | 0.6989 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.1502 | 17700 | 0.6889 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.1552 | 18290 | 0.6758 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.1602 | 18880 | 0.6799 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.1652 | 19470 | 0.7072 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.1702 | 20060 | 0.6748 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.1752 | 20650 | 0.6217 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.1802 | 21240 | 0.6249 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.1853 | 21830 | 0.626 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.1903 | 22420 | 0.619 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.1953 | 23010 | 0.593 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.2000 | 23568 | - | 0.5209 | 0.2559 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.2003 | 23600 | 0.6008 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.2053 | 24190 | 0.5831 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.2103 | 24780 | 0.5787 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.2153 | 25370 | 0.5746 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.2203 | 25960 | 0.5664 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.2253 | 26550 | 0.5637 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.2303 | 27140 | 0.5776 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.2353 | 27730 | 0.5617 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.2403 | 28320 | 0.5567 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.2453 | 28910 | 0.5296 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.2503 | 29500 | 0.5673 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.2554 | 30090 | 0.5625 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.2604 | 30680 | 0.515 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.2654 | 31270 | 0.5385 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.2704 | 31860 | 0.5408 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.2754 | 32450 | 0.4756 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.2804 | 33040 | 0.5208 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.2854 | 33630 | 0.5481 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.2904 | 34220 | 0.5212 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.2954 | 34810 | 0.5215 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.3000 | 35352 | - | 0.4023 | 0.2631 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.3004 | 35400 | 0.5206 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.3054 | 35990 | 0.4897 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.3104 | 36580 | 0.5032 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.3154 | 37170 | 0.5018 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.3204 | 37760 | 0.4943 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.3254 | 38350 | 0.4988 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.3305 | 38940 | 0.5003 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.3355 | 39530 | 0.531 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.3405 | 40120 | 0.4734 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.3455 | 40710 | 0.5096 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.3505 | 41300 | 0.4681 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.3555 | 41890 | 0.4864 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.3605 | 42480 | 0.4944 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.3655 | 43070 | 0.4972 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.3705 | 43660 | 0.4815 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.3755 | 44250 | 0.4757 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.3805 | 44840 | 0.4768 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.3855 | 45430 | 0.496 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.3905 | 46020 | 0.4858 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.3955 | 46610 | 0.4853 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.4000 | 47136 | - | 0.3836 | 0.2648 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.4005 | 47200 | 0.4829 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.4056 | 47790 | 0.466 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.4106 | 48380 | 0.4728 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.4156 | 48970 | 0.4387 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.4206 | 49560 | 0.4712 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.4256 | 50150 | 0.4876 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.4306 | 50740 | 0.4833 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.4356 | 51330 | 0.4837 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.4406 | 51920 | 0.481 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.4456 | 52510 | 0.4543 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.4506 | 53100 | 0.4849 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.4556 | 53690 | 0.4606 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.4606 | 54280 | 0.4453 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.4656 | 54870 | 0.4459 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.4706 | 55460 | 0.4707 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.4757 | 56050 | 0.4757 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.4807 | 56640 | 0.4557 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.4857 | 57230 | 0.4425 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.4907 | 57820 | 0.4389 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.4957 | 58410 | 0.4603 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.5000 | 58920 | - | 0.3547 | 0.2686 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.5007 | 59000 | 0.4599 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.5057 | 59590 | 0.4437 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.5107 | 60180 | 0.4552 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.5157 | 60770 | 0.456 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.5207 | 61360 | 0.4585 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.5257 | 61950 | 0.4645 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.5307 | 62540 | 0.4293 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.5357 | 63130 | 0.4604 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.5407 | 63720 | 0.4666 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.5457 | 64310 | 0.4552 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.5508 | 64900 | 0.4577 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.5558 | 65490 | 0.4545 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.5608 | 66080 | 0.436 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.5658 | 66670 | 0.4267 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.5708 | 67260 | 0.4182 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.5758 | 67850 | 0.4568 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.5808 | 68440 | 0.473 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.5858 | 69030 | 0.4714 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.5908 | 69620 | 0.4496 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.5958 | 70210 | 0.4432 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.6000 | 70704 | - | 0.3622 | 0.2683 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.6008 | 70800 | 0.4509 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.6058 | 71390 | 0.4336 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.6108 | 71980 | 0.4545 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.6158 | 72570 | 0.431 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.6209 | 73160 | 0.4188 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.6259 | 73750 | 0.4425 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.6309 | 74340 | 0.4389 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.6359 | 74930 | 0.4391 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.6409 | 75520 | 0.4402 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.6459 | 76110 | 0.4481 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.6509 | 76700 | 0.447 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.6559 | 77290 | 0.436 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.6609 | 77880 | 0.4525 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.6659 | 78470 | 0.4591 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.6709 | 79060 | 0.4305 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.6759 | 79650 | 0.4285 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.6809 | 80240 | 0.4166 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.6859 | 80830 | 0.4156 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.6909 | 81420 | 0.4186 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.6960 | 82010 | 0.4154 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.7000 | 82488 | - | 0.3448 | 0.2696 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.7010 | 82600 | 0.4436 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.7060 | 83190 | 0.4117 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.7110 | 83780 | 0.4372 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.7160 | 84370 | 0.4157 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.7210 | 84960 | 0.4293 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.7260 | 85550 | 0.4382 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.7310 | 86140 | 0.4494 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.7360 | 86730 | 0.458 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.7410 | 87320 | 0.4291 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.7460 | 87910 | 0.456 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.7510 | 88500 | 0.4279 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.7560 | 89090 | 0.4443 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.7610 | 89680 | 0.4131 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.7661 | 90270 | 0.4332 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.7711 | 90860 | 0.4337 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.7761 | 91450 | 0.4193 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.7811 | 92040 | 0.4314 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.7861 | 92630 | 0.4375 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.7911 | 93220 | 0.4281 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.7961 | 93810 | 0.4345 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | **0.8** | **94272** | **-** | **0.3434** | **0.2691** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | | 0.8011 | 94400 | 0.4237 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.8061 | 94990 | 0.4377 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.8111 | 95580 | 0.4336 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.8161 | 96170 | 0.4599 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.8211 | 96760 | 0.4177 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.8261 | 97350 | 0.4319 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.8311 | 97940 | 0.4174 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.8361 | 98530 | 0.4282 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.8412 | 99120 | 0.4319 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.8462 | 99710 | 0.4345 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.8512 | 100300 | 0.4394 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.8562 | 100890 | 0.4608 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.8612 | 101480 | 0.4511 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.8662 | 102070 | 0.4295 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.8712 | 102660 | 0.424 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.8762 | 103250 | 0.4469 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.8812 | 103840 | 0.4428 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.8862 | 104430 | 0.439 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.8912 | 105020 | 0.4397 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.8962 | 105610 | 0.3906 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.9000 | 106056 | - | 0.3468 | 0.2692 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.9012 | 106200 | 0.4241 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.9062 | 106790 | 0.4288 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.9113 | 107380 | 0.4359 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.9163 | 107970 | 0.4419 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.9213 | 108560 | 0.4205 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.9263 | 109150 | 0.4511 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.9313 | 109740 | 0.4195 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.9363 | 110330 | 0.4258 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.9413 | 110920 | 0.4505 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.9463 | 111510 | 0.4414 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.9513 | 112100 | 0.4297 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.9563 | 112690 | 0.4449 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.9613 | 113280 | 0.4471 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.9663 | 113870 | 0.4303 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.9713 | 114460 | 0.4159 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.9763 | 115050 | 0.4321 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.9813 | 115640 | 0.4545 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.9864 | 116230 | 0.4449 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.9914 | 116820 | 0.4634 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.9964 | 117410 | 0.4552 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 1.0 | 117838 | - | - | - | 0.6020 | 0.8560 | 0.8555 | 0.7590 | 0.7556 | 0.8799 | 0.8823 | 0.7777 | 0.7730 | 0.8363 | 0.8371 | 0.7227 | 0.7444 | 0.9170 | 0.9015 | 0.8358 | 0.8566 | 0.7686 | 0.7699 | 0.6492 | 0.6502 | 0.6325 | 0.6193 | 0.6138 | 0.5915 | * The bold row denotes the saved checkpoint.
### Framework Versions - Python: 3.11.10 - Sentence Transformers: 3.3.0 - Transformers: 4.46.3 - PyTorch: 2.5.1+cu124 - Accelerate: 1.1.1 - Datasets: 3.1.0 - Tokenizers: 0.20.3 ## Citation ### BibTeX #### Sentence Transformers ```bibtex @inproceedings{reimers-2019-sentence-bert, title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", author = "Reimers, Nils and Gurevych, Iryna", booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", month = "11", year = "2019", publisher = "Association for Computational Linguistics", url = "https://arxiv.org/abs/1908.10084", } ```