metadata
base_model: sentence-transformers/all-mpnet-base-v2
datasets: []
language: []
library_name: sentence-transformers
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:80
- loss:MultipleNegativesRankingLoss
widget:
- source_sentence: What is the short title of this Act?
sentences:
- The Banking (Amendment) Act, No. 24 of 2024
- >-
The Central Bank can assess whether granting a license is in the
interest of the banking system and national economy
- >-
They must conduct offshore banking business only with non-residents and
certain residents as determined by the Central Bank
- source_sentence: >-
What new power does the Central Bank have regarding excess accommodations
to related parties?
sentences:
- Accommodation and all financial investments
- >-
The Central Bank can assess a bank's track record for operating with
good governance and integrity
- >-
The Central Bank may require the bank to deduct such excess from
regulatory capital calculations
- source_sentence: >-
What new requirement is there for licensed commercial banks regarding the
disclosure of audit information?
sentences:
- >-
Auditors must immediately report findings that could materially affect
the bank's safety and soundness
- >-
They must prepare financial statements that represent a true and fair
view of the bank's financial position
- >-
To the Governor, Deputy Governor, Assistant Governor, Director of Bank
Supervision, or other high-ranking officers
- source_sentence: >-
Who approves the appointment, election, or nomination of directors for
licensed commercial banks?
sentences:
- The Central Bank
- >-
The Central Bank can designate foreign currencies for offshore banking
business
- >-
The Central Bank can request information or documents from any person
other than a licensed bank
- source_sentence: >-
What new power does the Central Bank have regarding the number of
directors on a bank's board?
sentences:
- Within fifteen days of becoming aware of such facts
- Transparency in ownership structure and beneficial ownership
- >-
The Central Bank can determine the number of board members, which shall
not be less than seven
SentenceTransformer based on sentence-transformers/all-mpnet-base-v2
This is a sentence-transformers model finetuned from sentence-transformers/all-mpnet-base-v2. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
Model Details
Model Description
- Model Type: Sentence Transformer
- Base model: sentence-transformers/all-mpnet-base-v2
- Maximum Sequence Length: 384 tokens
- Output Dimensionality: 768 tokens
- Similarity Function: Cosine Similarity
Model Sources
- Documentation: Sentence Transformers Documentation
- Repository: Sentence Transformers on GitHub
- Hugging Face: Sentence Transformers on Hugging Face
Full Model Architecture
SentenceTransformer(
(0): Transformer({'max_seq_length': 384, 'do_lower_case': False}) with Transformer model: MPNetModel
(1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
(2): Normalize()
)
Usage
Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
pip install -U sentence-transformers
Then you can load this model and run inference.
from sentence_transformers import SentenceTransformer
# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
"What new power does the Central Bank have regarding the number of directors on a bank's board?",
'The Central Bank can determine the number of board members, which shall not be less than seven',
'Within fifteen days of becoming aware of such facts',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]
# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
Training Details
Training Dataset
Unnamed Dataset
- Size: 80 training samples
- Columns:
query
andanswer
- Approximate statistics based on the first 1000 samples:
query answer type string string details - min: 8 tokens
- mean: 16.82 tokens
- max: 22 tokens
- min: 3 tokens
- mean: 16.85 tokens
- max: 28 tokens
- Samples:
query answer What new power does the Central Bank have regarding corrective action for misleading disclosures?
The Central Bank can require banks to publish corrected disclosure statements
What new requirement is there for licensed commercial banks regarding corporate governance?
The Board of Directors is ultimately responsible for ensuring compliance with laws and sound banking practices
Who is responsible for overseeing the management of a licensed commercial bank?
The Board of Directors
- Loss:
MultipleNegativesRankingLoss
with these parameters:{ "scale": 20.0, "similarity_fct": "cos_sim" }
Evaluation Dataset
Unnamed Dataset
- Size: 20 evaluation samples
- Columns:
query
andanswer
- Approximate statistics based on the first 1000 samples:
query answer type string string details - min: 11 tokens
- mean: 17.4 tokens
- max: 23 tokens
- min: 5 tokens
- mean: 17.2 tokens
- max: 29 tokens
- Samples:
query answer What new power does the Central Bank have regarding the assessment of bank license applications?
The Central Bank can assess whether granting a license is in the interest of the banking system and national economy
What new definition is provided for 'unlawful activity' in the Act?
It has the same meaning as in the Prevention of Money Laundering Act, No. 5 of 2006
What new requirement is there for licensed commercial banks regarding offshore banking?
They must conduct offshore banking business only with non-residents and certain residents as determined by the Central Bank
- Loss:
MultipleNegativesRankingLoss
with these parameters:{ "scale": 20.0, "similarity_fct": "cos_sim" }
Training Hyperparameters
Non-Default Hyperparameters
eval_strategy
: epochper_device_eval_batch_size
: 16learning_rate
: 3e-05warmup_ratio
: 0.1fp16
: Trueload_best_model_at_end
: Trueddp_find_unused_parameters
: False
All Hyperparameters
Click to expand
overwrite_output_dir
: Falsedo_predict
: Falseeval_strategy
: epochprediction_loss_only
: Trueper_device_train_batch_size
: 8per_device_eval_batch_size
: 16per_gpu_train_batch_size
: Noneper_gpu_eval_batch_size
: Nonegradient_accumulation_steps
: 1eval_accumulation_steps
: Nonetorch_empty_cache_steps
: Nonelearning_rate
: 3e-05weight_decay
: 0.0adam_beta1
: 0.9adam_beta2
: 0.999adam_epsilon
: 1e-08max_grad_norm
: 1.0num_train_epochs
: 3max_steps
: -1lr_scheduler_type
: linearlr_scheduler_kwargs
: {}warmup_ratio
: 0.1warmup_steps
: 0log_level
: passivelog_level_replica
: warninglog_on_each_node
: Truelogging_nan_inf_filter
: Truesave_safetensors
: Truesave_on_each_node
: Falsesave_only_model
: Falserestore_callback_states_from_checkpoint
: Falseno_cuda
: Falseuse_cpu
: Falseuse_mps_device
: Falseseed
: 42data_seed
: Nonejit_mode_eval
: Falseuse_ipex
: Falsebf16
: Falsefp16
: Truefp16_opt_level
: O1half_precision_backend
: autobf16_full_eval
: Falsefp16_full_eval
: Falsetf32
: Nonelocal_rank
: 0ddp_backend
: Nonetpu_num_cores
: Nonetpu_metrics_debug
: Falsedebug
: []dataloader_drop_last
: Falsedataloader_num_workers
: 0dataloader_prefetch_factor
: Nonepast_index
: -1disable_tqdm
: Falseremove_unused_columns
: Truelabel_names
: Noneload_best_model_at_end
: Trueignore_data_skip
: Falsefsdp
: []fsdp_min_num_params
: 0fsdp_config
: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}fsdp_transformer_layer_cls_to_wrap
: Noneaccelerator_config
: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}deepspeed
: Nonelabel_smoothing_factor
: 0.0optim
: adamw_torchoptim_args
: Noneadafactor
: Falsegroup_by_length
: Falselength_column_name
: lengthddp_find_unused_parameters
: Falseddp_bucket_cap_mb
: Noneddp_broadcast_buffers
: Falsedataloader_pin_memory
: Truedataloader_persistent_workers
: Falseskip_memory_metrics
: Trueuse_legacy_prediction_loop
: Falsepush_to_hub
: Falseresume_from_checkpoint
: Nonehub_model_id
: Nonehub_strategy
: every_savehub_private_repo
: Falsehub_always_push
: Falsegradient_checkpointing
: Falsegradient_checkpointing_kwargs
: Noneinclude_inputs_for_metrics
: Falseeval_do_concat_batches
: Truefp16_backend
: autopush_to_hub_model_id
: Nonepush_to_hub_organization
: Nonemp_parameters
:auto_find_batch_size
: Falsefull_determinism
: Falsetorchdynamo
: Noneray_scope
: lastddp_timeout
: 1800torch_compile
: Falsetorch_compile_backend
: Nonetorch_compile_mode
: Nonedispatch_batches
: Nonesplit_batches
: Noneinclude_tokens_per_second
: Falseinclude_num_input_tokens_seen
: Falseneftune_noise_alpha
: Noneoptim_target_modules
: Nonebatch_eval_metrics
: Falseeval_on_start
: Falseeval_use_gather_object
: Falsebatch_sampler
: batch_samplermulti_dataset_batch_sampler
: proportional
Training Logs
Epoch | Step | Training Loss | loss |
---|---|---|---|
0.2 | 2 | 1.1686 | - |
0.4 | 4 | 1.0061 | - |
0.6 | 6 | 0.8377 | - |
0.8 | 8 | 0.8121 | - |
1.0 | 10 | 0.267 | 0.8161 |
1.2 | 12 | 0.3282 | - |
1.4 | 14 | 0.1546 | - |
1.6 | 16 | 0.3546 | - |
1.8 | 18 | 0.0544 | - |
2.0 | 20 | 0.0271 | 0.6723 |
2.2 | 22 | 0.1496 | - |
2.4 | 24 | 0.0569 | - |
2.6 | 26 | 0.1682 | - |
2.8 | 28 | 0.0452 | - |
3.0 | 30 | 0.0261 | 0.6644 |
Framework Versions
- Python: 3.10.14
- Sentence Transformers: 3.0.1
- Transformers: 4.43.1
- PyTorch: 2.3.0
- Accelerate: 0.32.0
- Datasets: 2.19.1
- Tokenizers: 0.19.1
Citation
BibTeX
Sentence Transformers
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
MultipleNegativesRankingLoss
@misc{henderson2017efficient,
title={Efficient Natural Language Response Suggestion for Smart Reply},
author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
year={2017},
eprint={1705.00652},
archivePrefix={arXiv},
primaryClass={cs.CL}
}