---
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:1290285
- loss:TemperatureScaledCosineSimilarityLoss
widget:
- source_sentence: garden area
sentences:
- minimalistisk design
- grönområde i närheten
- vaaleat puusävyt
- source_sentence: ainutlaatuinen rakennus
sentences:
- sovrum med parkettgolv
- light wood flooring
- modern fixtures
- source_sentence: förortsområde
sentences:
- lush greenery
- well
- hyvin valaistu
- source_sentence: single-level
sentences:
- lähellä luontoa
- trappor till entrén
- light colored sofa
- source_sentence: gravel parking
sentences:
- asfalterad uppfart
- rauhallinen naapurusto
- asfalterad uppfart
pipeline_tag: sentence-similarity
library_name: sentence-transformers
metrics:
- pearson_cosine
- spearman_cosine
model-index:
- name: SentenceTransformer
results:
- task:
type: semantic-similarity
name: Semantic Similarity
dataset:
name: trait eval
type: trait-eval
metrics:
- type: pearson_cosine
value: 0.7845309961698034
name: Pearson Cosine
- type: spearman_cosine
value: 0.8182212433993569
name: Spearman Cosine
---
# SentenceTransformer
This is a [sentence-transformers](https://www.SBERT.net) model trained. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
## Model Details
### Model Description
- **Model Type:** Sentence Transformer
- **Maximum Sequence Length:** 128 tokens
- **Output Dimensionality:** 384 dimensions
- **Similarity Function:** Cosine Similarity
### Model Sources
- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co./models?library=sentence-transformers)
### Full Model Architecture
```
SentenceTransformer(
(0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: BertModel
(1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)
```
## Usage
### Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
```bash
pip install -U sentence-transformers
```
Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer
# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
'gravel parking',
'asfalterad uppfart',
'rauhallinen naapurusto',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]
# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```
## Evaluation
### Metrics
#### Semantic Similarity
* Dataset: `trait-eval`
* Evaluated with [EmbeddingSimilarityEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.EmbeddingSimilarityEvaluator)
| Metric | Value |
|:--------------------|:-----------|
| pearson_cosine | 0.7845 |
| **spearman_cosine** | **0.8182** |
## Training Details
### Training Dataset
#### Unnamed Dataset
* Size: 1,290,285 training samples
* Columns: sentence_0, sentence_1, and label
* Approximate statistics based on the first 1000 samples:
| | sentence_0 | sentence_1 | label |
|:--------|:---------------------------------------------------------------------------------|:---------------------------------------------------------------------------------|:---------------------------------------------------------------|
| type | string | string | float |
| details | - min: 3 tokens
- mean: 5.36 tokens
- max: 12 tokens
| - min: 3 tokens
- mean: 5.47 tokens
- max: 11 tokens
| - min: 0.0
- mean: 0.34
- max: 0.9
|
* Samples:
| sentence_0 | sentence_1 | label |
|:------------------------------------|:----------------------------|:-----------------|
| kerrostalo | puuaita | 0.1 |
| corner unit | large yard | 0.3 |
| easy access to highway | oma pysäköinti | 0.3 |
* Loss: __main__.TemperatureScaledCosineSimilarityLoss with these parameters:
```json
{
"loss_fct": "torch.nn.modules.loss.MSELoss"
}
```
### Training Hyperparameters
#### Non-Default Hyperparameters
- `eval_strategy`: steps
- `per_device_train_batch_size`: 64
- `per_device_eval_batch_size`: 64
- `num_train_epochs`: 5
- `multi_dataset_batch_sampler`: round_robin
#### All Hyperparameters
Click to expand
- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: steps
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 64
- `per_device_eval_batch_size`: 64
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 1
- `eval_accumulation_steps`: None
- `torch_empty_cache_steps`: None
- `learning_rate`: 5e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1
- `num_train_epochs`: 5
- `max_steps`: -1
- `lr_scheduler_type`: linear
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.0
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: False
- `fp16`: False
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: None
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: False
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: None
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `include_for_metrics`: []
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`:
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `dispatch_batches`: None
- `split_batches`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `eval_on_start`: False
- `use_liger_kernel`: False
- `eval_use_gather_object`: False
- `average_tokens_across_devices`: False
- `prompts`: None
- `batch_sampler`: batch_sampler
- `multi_dataset_batch_sampler`: round_robin
Click to expand
| Epoch | Step | Training Loss | trait-eval_spearman_cosine |
|:------:|:------:|:-------------:|:--------------------------:|
| 0.0248 | 500 | 0.1261 | - |
| 0.0496 | 1000 | 0.1155 | 0.2026 |
| 0.0744 | 1500 | 0.119 | - |
| 0.0992 | 2000 | 0.1193 | 0.2516 |
| 0.1240 | 2500 | 0.118 | - |
| 0.1488 | 3000 | 0.1151 | 0.2638 |
| 0.1736 | 3500 | 0.1152 | - |
| 0.1984 | 4000 | 0.1093 | 0.2700 |
| 0.2232 | 4500 | 0.1077 | - |
| 0.2480 | 5000 | 0.109 | 0.2942 |
| 0.2728 | 5500 | 0.1015 | - |
| 0.2976 | 6000 | 0.1059 | 0.3171 |
| 0.3224 | 6500 | 0.1052 | - |
| 0.3472 | 7000 | 0.1042 | 0.3114 |
| 0.3720 | 7500 | 0.1006 | - |
| 0.3968 | 8000 | 0.1007 | 0.3328 |
| 0.4216 | 8500 | 0.1013 | - |
| 0.4464 | 9000 | 0.0938 | 0.3407 |
| 0.4712 | 9500 | 0.0971 | - |
| 0.4960 | 10000 | 0.0976 | 0.3430 |
| 0.5208 | 10500 | 0.0945 | - |
| 0.5456 | 11000 | 0.0937 | 0.3603 |
| 0.5704 | 11500 | 0.0919 | - |
| 0.5952 | 12000 | 0.0893 | 0.3916 |
| 0.6200 | 12500 | 0.0904 | - |
| 0.6448 | 13000 | 0.0866 | 0.3931 |
| 0.6696 | 13500 | 0.087 | - |
| 0.6944 | 14000 | 0.0848 | 0.4178 |
| 0.7192 | 14500 | 0.087 | - |
| 0.7440 | 15000 | 0.0827 | 0.4218 |
| 0.7688 | 15500 | 0.0851 | - |
| 0.7936 | 16000 | 0.0807 | 0.4414 |
| 0.8184 | 16500 | 0.0803 | - |
| 0.8432 | 17000 | 0.0803 | 0.4526 |
| 0.8680 | 17500 | 0.0784 | - |
| 0.8928 | 18000 | 0.0776 | 0.4592 |
| 0.9176 | 18500 | 0.0761 | - |
| 0.9424 | 19000 | 0.0739 | 0.4856 |
| 0.9672 | 19500 | 0.0724 | - |
| 0.9920 | 20000 | 0.0738 | 0.4928 |
| 1.0 | 20161 | - | 0.4863 |
| 1.0168 | 20500 | 0.0707 | - |
| 1.0416 | 21000 | 0.0699 | 0.4950 |
| 1.0664 | 21500 | 0.0637 | - |
| 1.0912 | 22000 | 0.0677 | 0.5000 |
| 1.1160 | 22500 | 0.0638 | - |
| 1.1408 | 23000 | 0.0653 | 0.5306 |
| 1.1656 | 23500 | 0.0661 | - |
| 1.1904 | 24000 | 0.0679 | 0.5292 |
| 1.2152 | 24500 | 0.0618 | - |
| 1.2400 | 25000 | 0.0616 | 0.5253 |
| 1.2648 | 25500 | 0.0626 | - |
| 1.2896 | 26000 | 0.0624 | 0.5422 |
| 1.3144 | 26500 | 0.0613 | - |
| 1.3392 | 27000 | 0.0623 | 0.5515 |
| 1.3640 | 27500 | 0.0601 | - |
| 1.3888 | 28000 | 0.0589 | 0.5606 |
| 1.4136 | 28500 | 0.06 | - |
| 1.4384 | 29000 | 0.0598 | 0.5774 |
| 1.4632 | 29500 | 0.0553 | - |
| 1.4880 | 30000 | 0.0596 | 0.5812 |
| 1.5128 | 30500 | 0.0547 | - |
| 1.5376 | 31000 | 0.0542 | 0.5956 |
| 1.5624 | 31500 | 0.0547 | - |
| 1.5872 | 32000 | 0.0545 | 0.6053 |
| 1.6120 | 32500 | 0.0525 | - |
| 1.6368 | 33000 | 0.0545 | 0.6137 |
| 1.6616 | 33500 | 0.0532 | - |
| 1.6864 | 34000 | 0.0525 | 0.6213 |
| 1.7112 | 34500 | 0.0529 | - |
| 1.7360 | 35000 | 0.0515 | 0.6152 |
| 1.7608 | 35500 | 0.0506 | - |
| 1.7856 | 36000 | 0.0502 | 0.6211 |
| 1.8104 | 36500 | 0.0513 | - |
| 1.8352 | 37000 | 0.0476 | 0.6344 |
| 1.8600 | 37500 | 0.0491 | - |
| 1.8848 | 38000 | 0.0485 | 0.6438 |
| 1.9096 | 38500 | 0.0488 | - |
| 1.9344 | 39000 | 0.0471 | 0.6591 |
| 1.9592 | 39500 | 0.049 | - |
| 1.9840 | 40000 | 0.0486 | 0.6692 |
| 2.0 | 40322 | - | 0.6619 |
| 2.0088 | 40500 | 0.044 | - |
| 2.0336 | 41000 | 0.0407 | 0.6729 |
| 2.0584 | 41500 | 0.0422 | - |
| 2.0832 | 42000 | 0.0427 | 0.6775 |
| 2.1080 | 42500 | 0.0425 | - |
| 2.1328 | 43000 | 0.043 | 0.6772 |
| 2.1576 | 43500 | 0.0426 | - |
| 2.1824 | 44000 | 0.04 | 0.6877 |
| 2.2072 | 44500 | 0.041 | - |
| 2.2320 | 45000 | 0.0422 | 0.6885 |
| 2.2568 | 45500 | 0.04 | - |
| 2.2816 | 46000 | 0.0399 | 0.6903 |
| 2.3064 | 46500 | 0.0414 | - |
| 2.3312 | 47000 | 0.0397 | 0.7008 |
| 2.3560 | 47500 | 0.0406 | - |
| 2.3808 | 48000 | 0.0395 | 0.7004 |
| 2.4056 | 48500 | 0.0375 | - |
| 2.4304 | 49000 | 0.0391 | 0.6964 |
| 2.4552 | 49500 | 0.0378 | - |
| 2.4800 | 50000 | 0.0379 | 0.7092 |
| 2.5048 | 50500 | 0.0394 | - |
| 2.5296 | 51000 | 0.0399 | 0.7122 |
| 2.5544 | 51500 | 0.0357 | - |
| 2.5792 | 52000 | 0.0362 | 0.7170 |
| 2.6040 | 52500 | 0.0383 | - |
| 2.6288 | 53000 | 0.0396 | 0.7218 |
| 2.6536 | 53500 | 0.037 | - |
| 2.6784 | 54000 | 0.0372 | 0.7312 |
| 2.7032 | 54500 | 0.0372 | - |
| 2.7280 | 55000 | 0.0365 | 0.7303 |
| 2.7528 | 55500 | 0.0364 | - |
| 2.7776 | 56000 | 0.0356 | 0.7366 |
| 2.8024 | 56500 | 0.034 | - |
| 2.8272 | 57000 | 0.036 | 0.7415 |
| 2.8520 | 57500 | 0.035 | - |
| 2.8768 | 58000 | 0.0355 | 0.7402 |
| 2.9016 | 58500 | 0.0331 | - |
| 2.9264 | 59000 | 0.0345 | 0.7475 |
| 2.9512 | 59500 | 0.0345 | - |
| 2.9760 | 60000 | 0.0348 | 0.7489 |
| 3.0 | 60483 | - | 0.7500 |
| 3.0008 | 60500 | 0.0351 | - |
| 3.0256 | 61000 | 0.0294 | 0.7537 |
| 3.0504 | 61500 | 0.0297 | - |
| 3.0752 | 62000 | 0.0301 | 0.7557 |
| 3.1000 | 62500 | 0.0311 | - |
| 3.1248 | 63000 | 0.0314 | 0.7628 |
| 3.1496 | 63500 | 0.0288 | - |
| 3.1744 | 64000 | 0.0311 | 0.7713 |
| 3.1992 | 64500 | 0.03 | - |
| 3.2240 | 65000 | 0.0307 | 0.7728 |
| 3.2488 | 65500 | 0.0321 | - |
| 3.2736 | 66000 | 0.028 | 0.7726 |
| 3.2984 | 66500 | 0.031 | - |
| 3.3232 | 67000 | 0.0283 | 0.7717 |
| 3.3480 | 67500 | 0.0302 | - |
| 3.3728 | 68000 | 0.0302 | 0.7808 |
| 3.3976 | 68500 | 0.0303 | - |
| 3.4224 | 69000 | 0.0285 | 0.7790 |
| 3.4472 | 69500 | 0.0304 | - |
| 3.4720 | 70000 | 0.0287 | 0.7856 |
| 3.4969 | 70500 | 0.0301 | - |
| 3.5217 | 71000 | 0.0285 | 0.7886 |
| 3.5465 | 71500 | 0.0295 | - |
| 3.5713 | 72000 | 0.0296 | 0.7899 |
| 3.5961 | 72500 | 0.0269 | - |
| 3.6209 | 73000 | 0.0278 | 0.7911 |
| 3.6457 | 73500 | 0.0299 | - |
| 3.6705 | 74000 | 0.0285 | 0.7898 |
| 3.6953 | 74500 | 0.0286 | - |
| 3.7201 | 75000 | 0.0281 | 0.7891 |
| 3.7449 | 75500 | 0.0308 | - |
| 3.7697 | 76000 | 0.0288 | 0.7893 |
| 3.7945 | 76500 | 0.0283 | - |
| 3.8193 | 77000 | 0.0264 | 0.7953 |
| 3.8441 | 77500 | 0.0265 | - |
| 3.8689 | 78000 | 0.0271 | 0.7942 |
| 3.8937 | 78500 | 0.0263 | - |
| 3.9185 | 79000 | 0.0278 | 0.7932 |
| 3.9433 | 79500 | 0.0258 | - |
| 3.9681 | 80000 | 0.028 | 0.7996 |
| 3.9929 | 80500 | 0.0284 | - |
| 4.0 | 80644 | - | 0.8017 |
| 4.0177 | 81000 | 0.0258 | 0.8022 |
| 4.0425 | 81500 | 0.027 | - |
| 4.0673 | 82000 | 0.0228 | 0.8034 |
| 4.0921 | 82500 | 0.0259 | - |
| 4.1169 | 83000 | 0.0257 | 0.8057 |
| 4.1417 | 83500 | 0.0248 | - |
| 4.1665 | 84000 | 0.025 | 0.8060 |
| 4.1913 | 84500 | 0.024 | - |
| 4.2161 | 85000 | 0.0267 | 0.8084 |
| 4.2409 | 85500 | 0.0244 | - |
| 4.2657 | 86000 | 0.0261 | 0.8058 |
| 4.2905 | 86500 | 0.0256 | - |
| 4.3153 | 87000 | 0.0252 | 0.8061 |
| 4.3401 | 87500 | 0.0246 | - |
| 4.3649 | 88000 | 0.0243 | 0.8095 |
| 4.3897 | 88500 | 0.0243 | - |
| 4.4145 | 89000 | 0.0251 | 0.8113 |
| 4.4393 | 89500 | 0.0247 | - |
| 4.4641 | 90000 | 0.0239 | 0.8109 |
| 4.4889 | 90500 | 0.0248 | - |
| 4.5137 | 91000 | 0.0235 | 0.8129 |
| 4.5385 | 91500 | 0.0246 | - |
| 4.5633 | 92000 | 0.0231 | 0.8132 |
| 4.5881 | 92500 | 0.0254 | - |
| 4.6129 | 93000 | 0.0249 | 0.8140 |
| 4.6377 | 93500 | 0.0229 | - |
| 4.6625 | 94000 | 0.025 | 0.8143 |
| 4.6873 | 94500 | 0.0244 | - |
| 4.7121 | 95000 | 0.0227 | 0.8158 |
| 4.7369 | 95500 | 0.0223 | - |
| 4.7617 | 96000 | 0.0232 | 0.8166 |
| 4.7865 | 96500 | 0.024 | - |
| 4.8113 | 97000 | 0.0243 | 0.8170 |
| 4.8361 | 97500 | 0.0229 | - |
| 4.8609 | 98000 | 0.0243 | 0.8172 |
| 4.8857 | 98500 | 0.0223 | - |
| 4.9105 | 99000 | 0.0252 | 0.8176 |
| 4.9353 | 99500 | 0.0242 | - |
| 4.9601 | 100000 | 0.0221 | 0.8182 |
| 4.9849 | 100500 | 0.022 | - |
| 5.0 | 100805 | - | 0.8182 |