hi-di-hi-base

This is a sentence-transformers model finetuned from BAAI/bge-base-en-v1.5 on the json dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: BAAI/bge-base-en-v1.5
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • json
• Language: en
• License: apache-2.0

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("carsondial/hi-di-hi")
# Run inference
sentences = [
    'rui sagery bed and breakfast skye',
    "We LOVED our stay here! The have the best view of the water that we've ever seen! The sun setting from our bedroom window was amazing! We loved the way our room was decorated and the living room is especially gorgeous. We had a great night sleep on the very nice bedding. We would definitely spend a night here if we're ever in the area again. Thanks for such a wonderful place to stay.\n- Also Known As:\n- Ruisgarry Bed And Breakfast Breakish, Isle Of Skye",
    'Rennes, the gateway to Brittany\n|From Paris Cdg Airport||02:58||$32|\n|From Aix En Provence Tgv||06:02||$82|\n|From St Lo||02:00||$44|\n|From Tgv Haute Picardie||03:34||$98|\n|From Marne La Vallee||02:44||$32|\n|From Montreuil Sur Ill||00:16||$15|\n|From Montfort Sur Meu||00:12||$15|\n|From Messac Guipry||00:19||$15|\n|From St Brieuc||00:44||$24|\n|From Mont St Michel||01:15||$19|\n|From Le Mans||01:16||$29|\n|From St Gildas Des Boi||00:53||$24|\nGetting around From Rennes, you can travel to nearby cities and towns. Best price and travel duration below.\n|To Montfort Sur Meu||00:19||$15|\n|To Messac Guipry||00:38||$15|\n|To St Gildas Des Boi||01:05||$24|\n|To Mont St Michel||01:15||$19|\n|To Marne La Vallee||03:00||$32|\n|To St Brieuc||03:40||$23|\n|To Paris Cdg Airport||03:48||$32|\n|To Le Mans||03:48||$29|\n|To Aix En Provence Tgv||07:33||$148|\nWhile you\'re there\nRennes is the capital of Brittany region in France. The city of Rennes can be reached by the high speed TGV from Paris in 2 short hours. Rennes is the perfect base to visit other towns of Brittany, namely the Mont Saint-Michel and Saint-Malo. The city of Rennes is a dynamic city with over 50,000 students. The city has a rich architectural heritage, convivial restaurants and quality shops. When in Rennes, check street posters to track down a fest-noz – a time-honoured ball where generations meet to dance to traditional music and to drink and snack on authentic French delicacies. Take a night out to enjoy one of these rather than a nightclub and you’ll experience the real Rennes.\nIn the historic centre, explore the narrow, winding streets. You’ll discover colourful half-timbered houses dating back to the 15th century, the Cathedral Saint-Pierre and the city’s defensive heritage. In Place des Lices, is held the second largest food market in France every Saturday. Other notable landmarksin Rennes are: the two royal squares, the City Hall designed by Gabriel, the architect of Louis XV, the Parliament of Brittany, the Champs Libres Centre designed by Christian de Portzamparc, the Museum of Fine Arts, the Jardin du Thabor considered as one of the most beautiful French gardens, and the Saint-Georges swimming pool with mosaics designed by Odorico.\nIn Rennes you are most likely to have a taste of local specialities such as galettes and crêpes, pâtés, parlementins (pastries) and cider. If you are an amateur of fine cuisine, don’t miss the Festival Gourmand.\nRennes has plenty of markets to keep your roving eye occupied. Every Saturday the central food market in the Place des Lices near the Medieval Quarter sells huge varieties of French specialties. For good value clothes, games and make-up visit the Galleries Lafayette Mall. Between Place Sainte Anne and Place de la Mairie, there are numerous pedestrian street, ideal to shop.\nOffice de tourisme de Rennes\n11, rue Saint-Yves, 35064 RENNES CEDEX\nTel: +33 (0)2 99 67 11 11\nThe city’s tourist office has a helpful personnel, insider information and tips as well as handy maps and brochures. The city’s tourist office also arranges tours and excursions and is a focal point for local accommodation.\nTrains are a convenient way of reaching many towns and cities throughout Europe. The majority of cities in Europe have a minimum of one train station, while larger, more populated cities have two or more stations. Train stations, in general, are located in the heart of the city. Review the map below to get an idea of where the train station(s) in Rennes can be found.\nRennes railway station(s)\nRennes Train Station\nAddress: 19, Place de la gare BP 90527 35005 Rennes cedexOpening hours: Monday to Friday: 5am to 12:45am\nSaturday: 5am to 12:45am\nSunday: 6am to 12:45am Trains operating in and from this station:\nHigh speed trains (TGV), Regional trains (TER), Intercity trains\nSNCF rail agents, Assistance for disabled persons, Business lounges & Waiting rooms, Disabled facilities, Currency Exchange, Lost and found, Tourist Information Point, Toilets, Luggage storage, Wi-Fi Internet, ATM and Phone cabins.\nInteractive Rail Map\nMain cities in France\nanswered | Posted by marise f. | Comments 0 | Created: 05/08/2016\nanswered | Posted by Elizabeth W. | Comments 0 | Created: 18/10/2015\nanswered | Posted by HUEY L. | Comments 0 | Created: 22/07/2015\nanswered | Posted by PO-LIEN H. | Comments 0 | Created: 10/06/2015\nanswered | Posted by Michelle B. | Comments 0 | Created: 07/06/2015\n- "Fast and reliable" Winkie n. 30/01/2014\nIt was comfortable and relaxing. We enjoyed the scenery that you would not enjoy on the train. It was also on time. We could not have asked for more. I will travel by train in and around France.\n- "Train from Rennes to CDG " Barbara m. 13/12/2013\nI booked in advance and managed to obtain return first class tickets at a good price. The journeys were fast, smooth, warm, quiet and comfortable. I look forward to many more train journeys in the future.\n- "Paris to Rennes" Pauline l. 01/11/2013\nA very comfortable and efficient journey both going to Rennes and returning to Paris.\n- "first class seats are better " . 29/10/2013\nFirst class seats are better.. opt for that ! Online system is easy to use,pay, collect n off u go..\n- "Paris to Renne Trip" . 27/10/2013\nI rate our train trip from Paris to Rennes excellent! The experience was wonderful, the seats comfortable and the breakfast very good.\n- "TGV" Deon b. 12/10/2013\nThis an excellent fast\n- "An enjoyable trip" Erica a. 08/10/2013\nI enjoyed my trip on the TGV from Montparnasse to St Malo because I was able to relax, read my book and eat lunch, so there was no stress involved. The only thing I did have a bit of difficulty with was the ticket printer at the railway station in Montparnasse. I followed all the steps but the machine only printed my return ticket, not the ticket I needed that day! Luckily, however, the ticketing clerk in the office was able to find my booking and print my ticket for me, so all turned out well.\n- "booking N problems" Tatiana t. 07/10/2013\nMain problem was that with the booking reference Number of your email I couldn\'t print any of the SNCF\n- "Travelling in France" Elfride k. 01/10/2013\nThis was my first time travelling by train in France. At first it was a bit confusing for me, but that was due to the language barrier. I quickly got the hang of how the system works. It is very convenient and we travelled first class which is exactly as I expected it to be. Using the TGV was a great experience and I will recommend this mode of transport to all travellers. Elfride Kotze\n- "Paris - Rennes " Sulaiman a. 27/09/2013\nI enjoyed my travel trip to Rennes via TGV. But there was a 20 min. late when we want to take the train back to PAris. I hope it will be more punctuality next time.\n- "Comfortable" Yu Hsien c. 26/09/2013\nSoon and comfortable.\n- "TGV Paris - Rennes" Cooper Standard Polska c. 26/09/2013\n- "No train on Sunday " Thomas k. 11/09/2013\nI could not take a train that would get me to the airport on time for a flight back to the USA. I was flying on a Sunday so I understand that it may be a low travel day.\n- "Wonderful experitnce" Pin-hsun c. 05/09/2013\nWe went to Marseille from Paris by TGV train. We enjoyed the experience on the train.\n- "As my expectation" Shao-an l. 30/08/2013\nComfortable, fast, and convenient. If the price is cheaper would be much better.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

• Datasets: dim_768, dim_512, dim_256, dim_128 and dim_64
• Evaluated with InformationRetrievalEvaluator

Metric	dim_768	dim_512	dim_256	dim_128	dim_64
cosine_accuracy@1	0.5563	0.5565	0.5415	0.52	0.4745
cosine_accuracy@3	0.705	0.7015	0.6893	0.6687	0.628
cosine_accuracy@5	0.7575	0.7592	0.7442	0.7282	0.688
cosine_accuracy@10	0.8185	0.816	0.81	0.7955	0.7505
cosine_precision@1	0.5563	0.5565	0.5415	0.52	0.4745
cosine_precision@3	0.235	0.2338	0.2298	0.2229	0.2093
cosine_precision@5	0.1515	0.1518	0.1489	0.1457	0.1376
cosine_precision@10	0.0818	0.0816	0.081	0.0795	0.0751
cosine_recall@1	0.5563	0.5565	0.5415	0.52	0.4745
cosine_recall@3	0.705	0.7015	0.6893	0.6687	0.628
cosine_recall@5	0.7575	0.7592	0.7442	0.7282	0.688
cosine_recall@10	0.8185	0.816	0.81	0.7955	0.7505
cosine_ndcg@10	0.685	0.6842	0.6725	0.6539	0.6092
cosine_mrr@10	0.6425	0.6422	0.6289	0.6091	0.5642
cosine_map@100	0.6481	0.6478	0.6344	0.6148	0.5711

Training Details

Training Dataset

json

• Dataset: json
• Size: 36,000 training samples
• Columns: anchor and positive
• Approximate statistics based on the first 1000 samples:

  	anchor	positive
  type	string	string
  details	min: 4 tokens mean: 12.89 tokens max: 171 tokens	min: 107 tokens mean: 452.56 tokens max: 512 tokens

• Samples:

  anchor	positive
  seoul korean sheep cafe	Shamrock is less than a 5-minute drive from Mellieha Bay beach. It offers a sun terrace, air conditioning, and 2 balconies overlooking the Mediterranean Sea and Mellieha Cathedral. Wi-Fi is free. The apartment is set on the 2nd floor and features a 32” LCD TV with cable channels, a living/dining area with fully equipped kitchenette, and 2 balconies. It includes modern furniture and a washing machine. A bus to/from Valletta and Luqa Airport stops a few minutes’ walk from the property. Free parking in the street is available. Paradise Bay and ferries to Gozo leave from Cirkewwa Harbour, both 5 km away. A supermarket, pharmacy and restaurant are all a 2-minute walk from the Shamrock. The owners can also arrange for taxi service on request. Ocena od 174 opinie Adres: 15 Triq il Mithna L Qadima , Mellieħa MLH1345 (Mapa) Shamrock Flats - Zdjęcia Co mówią ludzie Very friendly landlord, mr Mario. He was very helpful in finding interesting activities on the island. We rented a car without navi ...
  fort kochi hotels near chinese fishing nets alternatively: hotels near chinese fishing nets fort kochi hotels fort kochi chinese fishing nets hotels with chinese fishing nets fort kochi fort kochi hot tub hotel fort kochi hotel with jacuzzi fort kochi hotels with hot tub hotels with hot tub fort kochi hot tub hotel kerala india hotels in kerala with hot tub kerala india hot tub hotel kochi india hot tub hotel	All bookings are confirmed instantly. No waiting, no email exchanges, no confusion. Our hotel reviews will help you find the best deal in the right location. • TODAY UP TO Do you have photos to share? Quick descriptionOffering a terrace and hot tub, Nena's Inn is set in Fort Kochi within 200 metres of the popular Chinese fishing nets and Santa Cruz Basilica. Free WiFi is offered and free private parking is available on site. A terrace or balcony are featured in certain rooms. Every room comes with a private bathroom fitted with a hot tub and bath. You will find a 24-hour front desk at the property. Bike hire is available at this homestay and the area is popular for cycling. Kumarakom is 46 km away. Hotel type: Guest accommodation Book now, pay later with FREE cancellation on many rooms! – Nena's Inn Check Rates & Availability Looking for the perfect accommodation in Cochin? Tell us your dates and we can help you find the best deal. * To view prices and availability, please enter your d...
  ramsray arms hotel dinner	Tel. (01561) 340 334 Fax. (01561) 340 500 Welcome to the Ramsay Arms Hotel & Restaurant, located in the picturesque village of Fettercairn at the foot of The Cairn o’ Mount. Our door is open to everyone, families with children, business travellers, for short and long holiday breaks or just do lunch, or evening dinner. The Ramsay Arms Hotel & Restaurant enjoys a beautiful and historic location at the gateway to the Scottish highlands, castles and whisky trails. The distillery in Fettercairn is the second oldest in Scotland. It has its own visitor center, which is a short stroll from the Ramsay Arms. Locally you can enjoy many activities including fishing on the River Esk or River Dye. Stalking, grouse or pheasant shooting can be arranged on the local estates of Fasque, Fettercairn and Gannochy. You really are spoilt for choice here, we even have 12 golf courses to choose from, which are close or a short drive away. The Ramsay can accommodate dining for up to 45 guests. Enjoy dining in t...

• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          768,
          512,
          256,
          128,
          64
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: epoch
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 16
• gradient_accumulation_steps: 16
• learning_rate: 2e-05
• num_train_epochs: 4
• lr_scheduler_type: cosine
• warmup_ratio: 0.1
• bf16: True
• tf32: True
• load_best_model_at_end: True
• optim: adamw_torch_fused
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: epoch
• prediction_loss_only: True
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 16
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 4
• max_steps: -1
• lr_scheduler_type: cosine
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: True
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional

Training Logs

Epoch	Step	Training Loss	dim_768_cosine_ndcg@10	dim_512_cosine_ndcg@10	dim_256_cosine_ndcg@10	dim_128_cosine_ndcg@10	dim_64_cosine_ndcg@10
0.1422	10	3.2548	-	-	-	-	-
0.2844	20	2.0118	-	-	-	-	-
0.4267	30	1.381	-	-	-	-	-
0.5689	40	0.9592	-	-	-	-	-
0.7111	50	0.9934	-	-	-	-	-
0.8533	60	0.8861	-	-	-	-	-
0.9956	70	0.8754	-	-	-	-	-
1.0	71	-	0.6795	0.6781	0.6633	0.6439	0.5925
1.1280	80	0.7392	-	-	-	-	-
1.2702	90	0.6947	-	-	-	-	-
1.4124	100	0.6543	-	-	-	-	-
1.5547	110	0.5582	-	-	-	-	-
1.6969	120	0.5893	-	-	-	-	-
1.8391	130	0.5662	-	-	-	-	-
1.9813	140	0.5773	-	-	-	-	-
2.0	142	-	0.6871	0.6856	0.6714	0.6526	0.6059
2.1138	150	0.5247	-	-	-	-	-
2.2560	160	0.4945	-	-	-	-	-
2.3982	170	0.4955	-	-	-	-	-
2.5404	180	0.4037	-	-	-	-	-
2.6827	190	0.4589	-	-	-	-	-
2.8249	200	0.4637	-	-	-	-	-
2.9671	210	0.4505	-	-	-	-	-
3.0	213	-	0.6859	0.6848	0.6725	0.6543	0.6084
3.0996	220	0.4105	-	-	-	-	-
3.2418	230	0.4262	-	-	-	-	-
3.384	240	0.4187	-	-	-	-	-
3.5262	250	0.3714	-	-	-	-	-
3.6684	260	0.3901	-	-	-	-	-
3.8107	270	0.4046	-	-	-	-	-
3.9529	280	0.4197	0.6850	0.6842	0.6725	0.6539	0.6092

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.10.12
• Sentence Transformers: 3.3.1
• Transformers: 4.47.1
• PyTorch: 2.5.1+cu124
• Accelerate: 1.2.1
• Datasets: 3.2.0
• Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}