README.md

---
base_model: indobenchmark/indobert-base-p1
datasets: []
language: []
library_name: sentence-transformers
metrics:
- pearson_cosine
- spearman_cosine
- pearson_manhattan
- spearman_manhattan
- pearson_euclidean
- spearman_euclidean
- pearson_dot
- spearman_dot
- pearson_max
- spearman_max
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:12800
- loss:ContrastiveTensionLoss
widget:
- source_sentence: Makalah ini diterbitkan dalam format online hanya oleh Metro International.
  sentences:
  - Liga ini berkembang dari tahun 1200 hingga 1500, dan terus menjadi semakin penting
    setelahnya.
  - Ini dirancang oleh orang lain selain WL Bottomley / William Lawrence Bottomley.
  - Lahan tersebut sekarang menjadi Cagar Alam Bentley Priory, sebuah Situs Kepentingan
    Ilmiah Khusus.
- source_sentence: Pengadilan menentang keputusan tahun 2010 dan kasus ini dilanjutkan
    sesuai dengan manfaatnya.
  sentences:
  - Gunung itu berada di Front Allegheny.
  - Stasiun St Albans Abbey adalah stasiun dalam perjalanan jalur ganda dari stasiun
    Watford Junction.
  - Pada tahun 2011, keluarga Penner tidak lagi menyebut rumah Habitatnya, rumah.
- source_sentence: Aku tidak jahat dalam hal ini.
  sentences:
  - Awalnya disetujui untuk onchocerciasis dan strongyloidiasis, Ivermectin sekarang
    disetujui oleh FDA untuk pedikulosis.
  - Lagu ini mencapai ARIA Singles Chart Top 100.
  - Bebaskan diri Anda dari permusuhan dan kemarahan untuk menunjukkan rasa hormat
    terhadap tubuh dan kehidupan Anda.
- source_sentence: Waktu pengiriman sangat cepat.
  sentences:
  - Dia kemudian bermain untuk South West Ham.
  - Qatar, bagaimanapun, tidak diminta untuk mengibarkan bendera Trucial yang ditentukan.
  - Sepasang pintu ini juga meredam suara dari luar.
- source_sentence: Dengan demikian, seorang model penutur harus mengolah representasi
    warna dalam konteks dan menghasilkan ujaran yang dapat membedakan warna sasaran
    dengan ujaran lainnya.
  sentences:
  - Dia bukan bagian dari American Institute of Architects.
  - Pada tahun 1975 VTL dibeli oleh Greyhound Lines, menjadi anak perusahaan.
  - Pada tanggal 24 April 2009, Forum Terbuka IBIS menyetujui versi 2.0.
model-index:
- name: SentenceTransformer based on indobenchmark/indobert-base-p1
  results:
  - task:
      type: semantic-similarity
      name: Semantic Similarity
    dataset:
      name: str dev
      type: str-dev
    metrics:
    - type: pearson_cosine
      value: 0.47668991144701395
      name: Pearson Cosine
    - type: spearman_cosine
      value: 0.48495339068233534
      name: Spearman Cosine
    - type: pearson_manhattan
      value: 0.5041035764250676
      name: Pearson Manhattan
    - type: spearman_manhattan
      value: 0.49270037559673846
      name: Spearman Manhattan
    - type: pearson_euclidean
      value: 0.5059182139447496
      name: Pearson Euclidean
    - type: spearman_euclidean
      value: 0.4915516775931335
      name: Spearman Euclidean
    - type: pearson_dot
      value: 0.2991963739133043
      name: Pearson Dot
    - type: spearman_dot
      value: 0.2630042391245101
      name: Spearman Dot
    - type: pearson_max
      value: 0.5059182139447496
      name: Pearson Max
    - type: spearman_max
      value: 0.49270037559673846
      name: Spearman Max
  - task:
      type: semantic-similarity
      name: Semantic Similarity
    dataset:
      name: str test
      type: str-test
    metrics:
    - type: pearson_cosine
      value: 0.47374249981827143
      name: Pearson Cosine
    - type: spearman_cosine
      value: 0.5083479438750005
      name: Spearman Cosine
    - type: pearson_manhattan
      value: 0.49828227586252527
      name: Pearson Manhattan
    - type: spearman_manhattan
      value: 0.4962152495999787
      name: Spearman Manhattan
    - type: pearson_euclidean
      value: 0.5006486050380166
      name: Pearson Euclidean
    - type: spearman_euclidean
      value: 0.49701891829828837
      name: Spearman Euclidean
    - type: pearson_dot
      value: 0.2573207350736585
      name: Pearson Dot
    - type: spearman_dot
      value: 0.24350607759185028
      name: Spearman Dot
    - type: pearson_max
      value: 0.5006486050380166
      name: Pearson Max
    - type: spearman_max
      value: 0.5083479438750005
      name: Spearman Max
---

# SentenceTransformer based on indobenchmark/indobert-base-p1

This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [indobenchmark/indobert-base-p1](https://huggingface.co./indobenchmark/indobert-base-p1). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

## Model Details

### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [indobenchmark/indobert-base-p1](https://huggingface.co./indobenchmark/indobert-base-p1) <!-- at revision c2cd0b51ddce6580eb35263b39b0a1e5fb0a39e2 -->
- **Maximum Sequence Length:** 32 tokens
- **Output Dimensionality:** 768 tokens
- **Similarity Function:** Cosine Similarity
<!-- - **Training Dataset:** Unknown -->
<!-- - **Language:** Unknown -->
<!-- - **License:** Unknown -->

### Model Sources

- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co./models?library=sentence-transformers)

### Full Model Architecture

```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 32, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)
```

## Usage

### Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

```bash
pip install -U sentence-transformers
```

Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("damand2061/negasibert-ct")
# Run inference
sentences = [
    'Dengan demikian, seorang model penutur harus mengolah representasi warna dalam konteks dan menghasilkan ujaran yang dapat membedakan warna sasaran dengan ujaran lainnya.',
    'Pada tahun 1975 VTL dibeli oleh Greyhound Lines, menjadi anak perusahaan.',
    'Pada tanggal 24 April 2009, Forum Terbuka IBIS menyetujui versi 2.0.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```

<!--
### Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details>
-->

<!--
### Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details>
-->

<!--
### Out-of-Scope Use

*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->

## Evaluation

### Metrics

#### Semantic Similarity
* Dataset: `str-dev`
* Evaluated with [<code>EmbeddingSimilarityEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.EmbeddingSimilarityEvaluator)

| Metric             | Value      |
|:-------------------|:-----------|
| pearson_cosine     | 0.4767     |
| spearman_cosine    | 0.485      |
| pearson_manhattan  | 0.5041     |
| spearman_manhattan | 0.4927     |
| pearson_euclidean  | 0.5059     |
| spearman_euclidean | 0.4916     |
| pearson_dot        | 0.2992     |
| spearman_dot       | 0.263      |
| pearson_max        | 0.5059     |
| **spearman_max**   | **0.4927** |

#### Semantic Similarity
* Dataset: `str-test`
* Evaluated with [<code>EmbeddingSimilarityEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.EmbeddingSimilarityEvaluator)

| Metric             | Value      |
|:-------------------|:-----------|
| pearson_cosine     | 0.4737     |
| spearman_cosine    | 0.5083     |
| pearson_manhattan  | 0.4983     |
| spearman_manhattan | 0.4962     |
| pearson_euclidean  | 0.5006     |
| spearman_euclidean | 0.497      |
| pearson_dot        | 0.2573     |
| spearman_dot       | 0.2435     |
| pearson_max        | 0.5006     |
| **spearman_max**   | **0.5083** |

<!--
## Bias, Risks and Limitations

*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->

<!--
### Recommendations

*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->

## Training Details

### Training Dataset

#### Unnamed Dataset


* Size: 12,800 training samples
* Columns: <code>sentence_0</code>, <code>sentence_1</code>, and <code>label</code>
* Approximate statistics based on the first 1000 samples:
  |         | sentence_0                                                                        | sentence_1                                                                        | label                                           |
  |:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:------------------------------------------------|
  | type    | string                                                                            | string                                                                            | int                                             |
  | details | <ul><li>min: 5 tokens</li><li>mean: 14.81 tokens</li><li>max: 32 tokens</li></ul> | <ul><li>min: 5 tokens</li><li>mean: 14.92 tokens</li><li>max: 32 tokens</li></ul> | <ul><li>0: ~87.50%</li><li>1: ~12.50%</li></ul> |
* Samples:
  | sentence_0                                                                                 | sentence_1                                                                        | label          |
  |:-------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:---------------|
  | <code>Warnanya tercermin pada corak dan lambang universitas kota tersebut.</code>          | <code>Warnanya tercermin pada corak dan lambang universitas kota tersebut.</code> | <code>1</code> |
  | <code>Pada awal tahun 2008, Ikerbasque menolak menugaskan Enrique Zuazua.</code>           | <code>Oh, ayolah, itu adil.</code>                                                | <code>0</code> |
  | <code>Pada tahun 2006, sebuah studi diselesaikan tentang prospek jalur Scarborough.</code> | <code>Jurnal Pendidikan Modern didirikan olehnya.</code>                          | <code>0</code> |
* Loss: [<code>ContrastiveTensionLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#contrastivetensionloss)

### Training Hyperparameters
#### Non-Default Hyperparameters

- `per_device_train_batch_size`: 64
- `per_device_eval_batch_size`: 64
- `num_train_epochs`: 5
- `multi_dataset_batch_sampler`: round_robin

#### All Hyperparameters
<details><summary>Click to expand</summary>

- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: no
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 64
- `per_device_eval_batch_size`: 64
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 1
- `eval_accumulation_steps`: None
- `torch_empty_cache_steps`: None
- `learning_rate`: 5e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1
- `num_train_epochs`: 5
- `max_steps`: -1
- `lr_scheduler_type`: linear
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.0
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: False
- `fp16`: False
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: None
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: False
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: False
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`: 
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `dispatch_batches`: None
- `split_batches`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `eval_on_start`: False
- `eval_use_gather_object`: False
- `batch_sampler`: batch_sampler
- `multi_dataset_batch_sampler`: round_robin

</details>

### Training Logs
| Epoch | Step | Training Loss | str-dev_spearman_max | str-test_spearman_max |
|:-----:|:----:|:-------------:|:--------------------:|:---------------------:|
| 1.0   | 200  | -             | 0.5009               | 0.5084                |
| 2.0   | 400  | -             | 0.4926               | 0.5025                |
| 2.5   | 500  | 2328.8573     | -                    | -                     |
| 3.0   | 600  | -             | 0.4909               | 0.5058                |
| 4.0   | 800  | -             | 0.4909               | 0.5064                |
| 5.0   | 1000 | 0.5625        | 0.4927               | 0.5083                |


### Framework Versions
- Python: 3.10.14
- Sentence Transformers: 3.0.1
- Transformers: 4.44.0
- PyTorch: 2.4.0
- Accelerate: 0.33.0
- Datasets: 2.21.0
- Tokenizers: 0.19.1

## Citation

### BibTeX

#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}
```

#### ContrastiveTensionLoss
```bibtex
@inproceedings{carlsson2021semantic,
    title={Semantic Re-tuning with Contrastive Tension},
    author={Fredrik Carlsson and Amaru Cuba Gyllensten and Evangelia Gogoulou and Erik Ylip{"a}{"a} Hellqvist and Magnus Sahlgren},
    booktitle={International Conference on Learning Representations},
    year={2021},
    url={https://openreview.net/forum?id=Ov_sMNau-PF}
}
```

<!--
## Glossary

*Clearly define terms in order to be accessible across audiences.*
-->

<!--
## Model Card Authors

*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->

<!--
## Model Card Contact

*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->