README.md

metadata

base_model: indobenchmark/indobert-base-p1
datasets: []
language: []
library_name: sentence-transformers
metrics:
  - pearson_cosine
  - spearman_cosine
  - pearson_manhattan
  - spearman_manhattan
  - pearson_euclidean
  - spearman_euclidean
  - pearson_dot
  - spearman_dot
  - pearson_max
  - spearman_max
pipeline_tag: sentence-similarity
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:12000
  - loss:MultipleNegativesSymmetricRankingLoss
widget:
  - source_sentence: Awalnya merupakan singkatan dari John's Macintosh Project.
    sentences:
      - >-
        Sebuah formasi yang terdiri dari sekitar 50 petugas Polisi Baltimore
        akhirnya menempatkan diri mereka di antara para perusuh dan milisi,
        memungkinkan Massachusetts ke-6 untuk melanjutkan ke Stasiun Camden.
      - Mengecat luka dapat melindungi dari jamur dan hama.
      - Dulunya merupakan singkatan dari John's Macintosh Project.
  - source_sentence: Boueiz berprofesi sebagai pengacara.
    sentences:
      - Mereka juga gagal mengembangkan Water Cooperation Quotient yang baru.
      - >-
        Pada Pemilu 1970, ia ikut serta dari Partai Persatuan Nasional namun
        dikalahkan.
      - Seorang pengacara berprofesi sebagai Boueiz.
  - source_sentence: Fakultas Studi Oriental memiliki seorang profesor.
    sentences:
      - >-
        Di tempat lain di New Mexico, LAHS terkadang dianggap sebagai sekolah
        untuk orang kaya.
      - >-
        Laporan lain juga menunjukkan kandungannya lebih rendah dari 0,1% di
        Australia.
      - Profesor tersebut merupakan bagian dari Fakultas Studi Oriental.
  - source_sentence: >-
      Hal ini terjadi di sejumlah negara, termasuk Ethiopia, Republik Demokratik
      Kongo, dan Afrika Selatan.
    sentences:
      - >-
        Hal ini diketahui terjadi di Eritrea, Ethiopia, Kongo, Tanzania, Namibia
        dan Afrika Selatan.
      - Gugus amil digantikan oleh gugus pentil.
      - Dan saya beritahu Anda sesuatu, itu tidak adil.
  - source_sentence: Ini adalah wilayah sosial-ekonomi yang lebih rendah.
    sentences:
      - >-
        Ini adalah bengkel perbaikan mobil terbaru yang masih beroperasi di
        kota.
      - >-
        Zelinsky hanya berteori bahwa tidak ada tiga bilangan bulat berurutan
        yang semuanya dapat difaktorkan ulang.
      - Ini adalah wilayah sosial-ekonomi yang lebih tinggi.
model-index:
  - name: SentenceTransformer based on indobenchmark/indobert-base-p1
    results:
      - task:
          type: semantic-similarity
          name: Semantic Similarity
        dataset:
          name: str dev
          type: str-dev
        metrics:
          - type: pearson_cosine
            value: 0.4607595775209637
            name: Pearson Cosine
          - type: spearman_cosine
            value: 0.48464707121470735
            name: Spearman Cosine
          - type: pearson_manhattan
            value: 0.5042489801555614
            name: Pearson Manhattan
          - type: spearman_manhattan
            value: 0.4966473433316482
            name: Spearman Manhattan
          - type: pearson_euclidean
            value: 0.5056344884375596
            name: Pearson Euclidean
          - type: spearman_euclidean
            value: 0.49855770055205806
            name: Spearman Euclidean
          - type: pearson_dot
            value: 0.3216463208701575
            name: Pearson Dot
          - type: spearman_dot
            value: 0.3018716261690138
            name: Spearman Dot
          - type: pearson_max
            value: 0.5056344884375596
            name: Pearson Max
          - type: spearman_max
            value: 0.49855770055205806
            name: Spearman Max
      - task:
          type: semantic-similarity
          name: Semantic Similarity
        dataset:
          name: str test
          type: str-test
        metrics:
          - type: pearson_cosine
            value: 0.4797624035508465
            name: Pearson Cosine
          - type: spearman_cosine
            value: 0.5041622737914666
            name: Spearman Cosine
          - type: pearson_manhattan
            value: 0.5006064051108505
            name: Pearson Manhattan
          - type: spearman_manhattan
            value: 0.49599768547328293
            name: Spearman Manhattan
          - type: pearson_euclidean
            value: 0.5010014604719228
            name: Pearson Euclidean
          - type: spearman_euclidean
            value: 0.4970249837224265
            name: Spearman Euclidean
          - type: pearson_dot
            value: 0.34489995684419983
            name: Pearson Dot
          - type: spearman_dot
            value: 0.3383462361299372
            name: Spearman Dot
          - type: pearson_max
            value: 0.5010014604719228
            name: Pearson Max
          - type: spearman_max
            value: 0.5041622737914666
            name: Spearman Max

SentenceTransformer based on indobenchmark/indobert-base-p1

This is a sentence-transformers model finetuned from indobenchmark/indobert-base-p1. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: indobenchmark/indobert-base-p1
• Maximum Sequence Length: 32 tokens
• Output Dimensionality: 768 tokens
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 32, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("damand2061/negasibert-mnrls")
# Run inference
sentences = [
    'Ini adalah wilayah sosial-ekonomi yang lebih rendah.',
    'Ini adalah wilayah sosial-ekonomi yang lebih tinggi.',
    'Zelinsky hanya berteori bahwa tidak ada tiga bilangan bulat berurutan yang semuanya dapat difaktorkan ulang.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Semantic Similarity

• Dataset: str-dev
• Evaluated with EmbeddingSimilarityEvaluator

Metric	Value
pearson_cosine	0.4608
spearman_cosine	0.4846
pearson_manhattan	0.5042
spearman_manhattan	0.4966
pearson_euclidean	0.5056
spearman_euclidean	0.4986
pearson_dot	0.3216
spearman_dot	0.3019
pearson_max	0.5056
spearman_max	0.4986

Semantic Similarity

• Dataset: str-test
• Evaluated with EmbeddingSimilarityEvaluator

Metric	Value
pearson_cosine	0.4798
spearman_cosine	0.5042
pearson_manhattan	0.5006
spearman_manhattan	0.496
pearson_euclidean	0.501
spearman_euclidean	0.497
pearson_dot	0.3449
spearman_dot	0.3383
pearson_max	0.501
spearman_max	0.5042

Training Details

Training Dataset

Unnamed Dataset

• Size: 12,000 training samples
• Columns: sentence_0 and sentence_1
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1
  type	string	string
  details	min: 5 tokens mean: 14.84 tokens max: 32 tokens	min: 5 tokens mean: 14.83 tokens max: 32 tokens

• Samples:

  sentence_0	sentence_1
  Pusat Peringatan Topan Gabungan (JTWC) juga mengeluarkan peringatan dalam kapasitas tidak resmi.	Pusat Peringatan Topan Gabungan (JTWC) hanya mengeluarkan peringatan dalam kapasitas yang tidak resmi.
  DNP komersial digunakan sebagai antiseptik dan pestisida bioakumulasi non-selektif.	DNP komersial tidak dapat digunakan sebagai antiseptik atau pestisida bioakumulasi non-selektif.
  Kuncian tulang belakang dan kuncian serviks diperbolehkan dan wajib dalam kompetisi jiu-jitsu Brasil IBJJF.	Kuncian tulang belakang dan kuncian serviks dilarang dalam kompetisi jiu-jitsu Brasil IBJJF.

• Loss: MultipleNegativesSymmetricRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 64
• per_device_eval_batch_size: 64
• num_train_epochs: 5
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 64
• per_device_eval_batch_size: 64
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 5
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: False
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• eval_use_gather_object: False
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin

Training Logs

Epoch	Step	Training Loss	str-dev_spearman_max	str-test_spearman_max
1.0	188	-	0.4912	0.5072
2.0	376	-	0.4940	0.5062
2.6596	500	0.0974	-	-
3.0	564	-	0.4942	0.5052
4.0	752	-	0.4962	0.5024
5.0	940	-	0.4986	0.5042

Framework Versions

• Python: 3.10.14
• Sentence Transformers: 3.0.1
• Transformers: 4.44.0
• PyTorch: 2.4.0
• Accelerate: 0.33.0
• Datasets: 2.21.0
• Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}