|
--- |
|
base_model: BAAI/bge-base-en-v1.5 |
|
datasets: [] |
|
language: |
|
- en |
|
library_name: sentence-transformers |
|
license: apache-2.0 |
|
metrics: |
|
- cosine_accuracy@1 |
|
- cosine_accuracy@3 |
|
- cosine_accuracy@5 |
|
- cosine_accuracy@10 |
|
- cosine_precision@1 |
|
- cosine_precision@3 |
|
- cosine_precision@5 |
|
- cosine_precision@10 |
|
- cosine_recall@1 |
|
- cosine_recall@3 |
|
- cosine_recall@5 |
|
- cosine_recall@10 |
|
- cosine_ndcg@10 |
|
- cosine_mrr@10 |
|
- cosine_map@100 |
|
pipeline_tag: sentence-similarity |
|
tags: |
|
- sentence-transformers |
|
- sentence-similarity |
|
- feature-extraction |
|
- generated_from_trainer |
|
- dataset_size:2231 |
|
- loss:MatryoshkaLoss |
|
- loss:MultipleNegativesRankingLoss |
|
widget: |
|
- source_sentence: The fact that no customer noticed this major migration to Amazon |
|
S3 Glacier Instant Retrieval was a big win for us. It was a seamless experience |
|
for end users, and we had no production issues during the entire migration. ” |
|
Contact Sales Greater than 99. 99% Outcome | Gaining Insights on AWS to Prioritize |
|
Business Needs 한국어 Snap migrated more than 2 exabytes of data—roughly equivalent |
|
to 1. 5 trillion media files—seamlessly to Amazon S3 Glacier Instant Retrieval |
|
from Amazon S3 Standard-IA. “The fact that no customer noticed this major migration |
|
to Amazon S3 Glacier Instant Retrieval was a big win for us,” says Manoharan. |
|
“It was a seamless experience for Snapchatters, and we had no production issues |
|
during the entire migration. ” As a result of the migration, the company saved |
|
tens of millions of dollars on storage. Snap has configured Amazon S3 in 20 AWS |
|
Regions around the world so that customers anywhere can retrieve data in milliseconds. |
|
The AWS Global Infrastructure is the most secure, extensive, and reliable Global |
|
Cloud Infrastructure for a business’s applications. The global reach of AWS lets |
|
Snap store media closer to the place where Snapchatters are creating it for optimal |
|
performance. Snap is also able to deliver content efficiently using Amazon CloudFront, |
|
a content delivery network service built for high performance, security, and availability. |
|
“We’ve been able to off-load all of the regionalization work and costs to AWS |
|
so that we can focus on developing new features,” says Manoharan. As a result, |
|
Snapchat continues to meet its quarterly cost-optimization goals. Overview | Opportunity |
|
| Solution | Outcome | AWS Services Used 2 exabytes Amazon Simple Storage Service |
|
(Amazon S3) is an object storage service offering industry-leading scalability, |
|
data availability, security, and performance. … In 2016, Snap migrated its data |
|
to AWS. “We chose to migrate to AWS because of its global reach, excellent performance, |
|
and competitive pricing that, in turn, gave us the ability to reinvest in our |
|
business,” says Vijay Manoharan, manager of the media delivery platform team at |
|
Snap. Amazon S3 Glacier Instant Retrieval is an archive storage class that delivers |
|
the lowest-cost storage for long-lived data that is rarely accessed and requires |
|
retrieval in milliseconds. AWS Services Used In 2017, Snap migrated one of the |
|
app’s most central features—Snapchat Stories—to Amazon DynamoDB, a fully managed, |
|
serverless, NoSQL database designed to run high-performance applications at virtually |
|
any scale. Using Amazon DynamoDB, the company experienced greater than 99. |
|
sentences: |
|
- How did Snap save tens of millions of dollars on storage as a result of migrating |
|
to Amazon S3 Glacier Instant Retrieval from Amazon S3 Standard-IA? |
|
- How has Panasonic Avionics Corporation leveraged Amazon Aurora MySQL-Compatible |
|
Edition and other AWS services to improve the reliability and scalability of its |
|
databases for in-flight entertainment and communications systems? |
|
- How does Ground Truth Plus ensure the quality of image and video captions generated |
|
by human annotators? |
|
- source_sentence: ” 中文 (繁體) Bahasa Indonesia Contact Sales Ρусский Customer Stories |
|
/ Software & Internet عربي 中文 (简体) Organizations of all sizes across all industries |
|
are transforming their businesses and delivering on their missions every day using |
|
AWS. Contact our experts and start your own AWS journey today. Outcome | Expanding |
|
Intelligent Features of Virtual Care Amazon Transcribe is an automatic speech |
|
recognition service that makes it easy to add speech to text capabilities to any |
|
application. Learn more » Learn more » It is critical that video visits are secure, |
|
responsive, and reliable. Using AWS helps us provide all this in a performant |
|
and scalable way. " Overview With the Amazon Chime SDK, builders can easily add |
|
real-time voice, video, and messaging powered by machine learning into their applications. |
|
Get Started Beyond traditional use cases, Salesforce is adding capabilities in |
|
medication-therapy management, connectivity for care coordinators, and other approaches |
|
for patient engagement. The company is developing a new feature that will expand |
|
its support of Virtual Care sessions to multiple participants, instead of just |
|
clinician and patient. This will facilitate care-team coordination with multiple |
|
parties in a single meeting. Using AWS, Salesforce circumvented the heavy lifting |
|
that would have been required to build and maintain a video-calling solution from |
|
scratch. Patients self-schedule virtual appointments, coordinate previsit activities, |
|
and conduct virtual visits in a HIPAA-compliant environment. A patient’s appointment |
|
request gets routed to Amazon Chime SDK. Clinicians then review a patient’s intake |
|
form and correlate the patient to a Virtual Care session using Amazon Chime SDK |
|
messaging, which connects providers and patients with secure, scalable messaging |
|
in their web and mobile applications. The Amazon Chime SDK control plane sends |
|
event notifications through a default event bus to Amazon EventBridge, a serverless |
|
event bus that helps organizations receive, filter, transform, route, and deliver |
|
events. Healthcare professionals deliver care over the internet in near real time, |
|
which has significantly reduced no-shows for appointments. “Using Amazon Chime |
|
SDK, we don’t have to worry about the mechanics of the video call,” Daftari says. |
|
“We can focus on features and functions that help differentiate our product in |
|
the marketplace, while also significantly improving our speed to launch. ” Salesforce |
|
further supports accessibility through embedding closed-captioning of video calls |
|
using Amazon Chime SDK live transcription. Amazon Chime SDK sends live audio streams |
|
to Amazon Transcribe, which automatically converts speech to text. Salesforce |
|
Health Cloud customers can use the live transcription capability to display subtitles, |
|
create meeting transcripts, or analyze content. |
|
sentences: |
|
- How did DB Energie use Amazon SageMaker and AWS to enhance the sustainability |
|
and reliability of its power grid operations? |
|
- How did Provectus assist Earth.com in enhancing the AI-powered image recognition |
|
capabilities of EarthSnap and reducing engineering heavy lifting through the implementation |
|
of end-to-end ML pipelines and managed MLOps platform? |
|
- How does Salesforce use AWS services such as Amazon Chime SDK and Amazon Transcribe |
|
to enhance their Virtual Care sessions for healthcare professionals and patients? |
|
- source_sentence: It’s been a great success. ” Overview 93% Validate technical skills |
|
and cloud expertise to grow your career and business. Learn more » Amazon Web |
|
Services (AWS) Education Programs collaborate with education institutions and |
|
the public sector to provide access for individuals to develop cloud computing |
|
and digital skills. To help graduates boost their employability, Staffordshire |
|
University worked with the AWS team to introduce cloud computing skills training |
|
and add cloud courses to its credit-bearing computer science modules. Staffordshire |
|
University offers courses through AWS Academy, which empowers higher education |
|
institutions to prepare students for industry-recognized certifications and careers. |
|
Since the university added AWS Academy courses to its curriculum in 2017, several |
|
hundred students have participated. Of those students, 93 percent have achieved |
|
employment within 6 months of graduation. Empowered students Türkçe Solution | |
|
Learning by Doing Using AWS Learner Labs English With AWS Academy, our students |
|
love that they’re not just taking theory lessons. They get to work in actual environments |
|
with real AWS tools. ” Next up, Staffordshire University is expanding on the success |
|
of its cloud courses by launching additional programs of study developed in collaboration |
|
with the AWS team. Staffordshire University and the AWS team designed these programs |
|
by "Working Backwards" — an Amazon process that encourages companies to brainstorm |
|
solutions by using a customer challenge as the starting point — from the cloud |
|
skills employers are currently seeking in the United Kingdom and across the global |
|
labor market. One of these programs, which launches in September 2022, is a cloud |
|
computing course that features both cloud computing and cybersecurity modules |
|
and will offer students more opportunities to discover what’s possible with the |
|
AWS Cloud. “What we want to encourage is for students to play with AWS services |
|
as well as build confidence with the tools,” says Dr. Champion. to learn remotely |
|
using any hardware and earn AWS Certifications Staffordshire University added |
|
cloud computing skills training to its curriculum using AWS Education Programs, |
|
helping 93 percent of participants find employment within 6 months of graduation. |
|
covering cloud skills AWS Certification during the AWS Educate University Challenge |
|
Deutsch of graduates find jobs within 6 months Tiếng Việt Italiano ไทย Outcome |
|
| Developing New Cloud Coursework About Staffordshire University Staffordshire |
|
University is a public research university in Staffordshire, England. Founded |
|
in 1914, the university serves over 15,000 students across three schools and four |
|
campuses. The United Kingdom has experienced a technology boom in recent years, |
|
with technology funding tripling in the first 6 months of 2021 compared to the |
|
same period in 2020. In particular, employers need professionals with cloud computing |
|
skills ranging from cloud development to machine learning and data analytics. |
|
To meet demand, Staffordshire University offers students their choice of six AWS |
|
courses covering these key skills and more. |
|
sentences: |
|
- How has the collaboration between Staffordshire University and the AWS team impacted |
|
the employability of graduates in the field of cloud computing? |
|
- How can the confidence scores be used to verify the accuracy of sentiment assignments |
|
in the sentiment_results_final table, especially for any dubious sentiment assignments? |
|
- How did migrating to AWS help Travian Games improve the stability and reliability |
|
of their game servers, and what impact did this have on their players' experience? |
|
- source_sentence: Contact our experts and start your own AWS journey today. customer |
|
and agent experience 2022 Overview WaFd Bank Transforms Contact Centers Using |
|
Conversational AI on AWS Customer Stories / Financial Services WaFd uses a data |
|
lake on AWS to store and analyze data from phone and chatbot conversations. “We’re |
|
getting incredible data from AWS through the conversational logs,” says Hubbard. |
|
“That has given us insights into what our customers are asking for so that we |
|
can add more self-service functionality. ” The data also gives WaFd more insight |
|
into call volumes, so the call center can better manage staff schedules. Opportunity |
|
| Using Amazon Lex to Implement an AI-Powered Contact Center Solution Türkçe English |
|
WaFd is a US retail and commercial bank with over 200 branches in eight states. |
|
In 2019, WaFd founded subsidiary Pike Street Labs, a fintech startup, to drive |
|
client-facing digital innovation for the bank. “Banks need to meet customers’ |
|
digital expectations,” says Dustin Hubbard, chief technology officer at WaFd Bank |
|
and Pike Street Labs. “Every year, customers expect more innovation because that’s |
|
what they see from new entrants or in other markets. ” Pike Street Labs redesigned |
|
WaFd’s online banking solution to provide personalized customer experiences and |
|
began tackling the bank’s customer care center. The company’s previous contact |
|
center solution used dated technology with limited features spread across disparate |
|
systems. This led to long wait times for customers and frustration for agents, |
|
who had to answer incoming calls without prior knowledge of what the customer |
|
needed. Agents also bore the burden of identifying fraudulent calls. WaFd needed |
|
a solution to improve both the customer and agent experiences. Previously, WaFd |
|
used two different systems in its customer care center to manage its voice and |
|
chat-based customer interactions, with no way for one system to recognize that |
|
an agent was busy on the other. Chat messages remained unanswered because agents |
|
would forget to sign in to the chat system. The company implemented chatbots and |
|
voice bots powered by Amazon Lex. Now, the call and chat systems are interoperable, |
|
and chats can be escalated to agent assisted calls when needed. When a call gets |
|
passed to an agent, the system also passes the full chat record and an analysis |
|
of the customer’s tone so that the agent is prepared to address the client’s needs |
|
and be empathetic toward the caller’s sentiment. WaFd worked with the AWS and |
|
Talkdesk teams to create and launch its new contact center solution in July 2022. |
|
sentences: |
|
- How did Yellow Class optimize its video files and improve performance using AWS |
|
services such as AWS Elemental MediaConvert? |
|
- How has FanDuel ensured the redundancy and reliability of its live video streams |
|
through the use of AWS Elemental MediaConnect and AWS Elemental MediaLive? |
|
- How did WaFd Bank use data from phone and chatbot conversations stored in a data |
|
lake on AWS to improve self-service functionality and better manage call center |
|
staff schedules? |
|
- source_sentence: 'Alternatively, you can run the inference via code. Here is one |
|
example written in Python, using the requests library: import requests url = "https://<YOUR_API_GATEWAY_ENDPOINT_ID>. |
|
execute-api. <YOUR_ENDPOINT_REGION>. amazonaws. com/prod/question?question=\"What |
|
is the color of my car now?\"&context=\"My car used to be blue but I painted red\"" |
|
response = requests. request("GET", url, headers=headers, data=payload) print(response. |
|
text) The code outputs a string similar to the following: ''{"score":0. 6947233080863953,"start":38,"end":41,"answer":"red"}'' |
|
If you are interested in knowing more about deploying Generative AI and large |
|
language models on AWS, check out here: Deploy Serverless Generative AI on AWS |
|
Lambda with OpenLLaMa Deploy large language models on AWS Inferentia2 using large |
|
model inference containers Clean up Inside the root directory of your repository, |
|
run the following code to clean up your resources: make destroy Conclusion In |
|
this post, we introduced how you can use Lambda to deploy your trained ML model |
|
using your preferred web application framework, such as FastAPI. We provided a |
|
detailed code repository that you can deploy, and you retain the flexibility of |
|
switching to whichever trained model artifacts you process. The performance can |
|
depend on how you implement and deploy the model. You are welcome to try it out |
|
yourself, and we’re excited to hear your feedback! About the Authors Tingyi Li |
|
is an Enterprise Solutions Architect from AWS based out in Stockholm, Sweden supporting |
|
the Nordics customers. She enjoys helping customers with the architecture, design, |
|
and development of cloud-optimized infrastructure solutions. She is specialized |
|
in AI and Machine Learning and is interested in empowering customers with intelligence |
|
in their AI/ML applications. In her spare time, she is also a part-time illustrator |
|
who writes novels and plays the piano. Demir Catovic is a Machine Learning Engineer |
|
from AWS based in Zurich, Switzerland. He engages with customers and helps them |
|
implement scalable and fully-functional ML applications. He is passionate about |
|
building and productionizing machine learning applications for customers and is |
|
always keen to explore around new trends and cutting-edge technologies in the |
|
AI/ML world. TAGS: Generative AI , Natural Language Processing Comments View Comments |
|
Resources Getting Started What''s New Blog Topics Amazon Comprehend Amazon Kendra |
|
Amazon Lex Amazon Polly Amazon Rekognition Amazon SageMaker Amazon Textract Follow |
|
Twitter Facebook LinkedIn Twitch Email Updates.' |
|
sentences: |
|
- How did ALTBalaji use AWS Elemental MediaLive to handle a tenfold increase in |
|
viewership during the live streaming of Lock Upp, and what insights did they gain |
|
from this experience? |
|
- How has PayEye been able to accelerate their development process and enter the |
|
production phase within a few months using AWS services, and what impact has this |
|
had on their recruitment efforts and team focus? |
|
- How can Lambda be used to deploy trained ML models using a preferred web application |
|
framework? |
|
model-index: |
|
- name: BGE base Financial Matryoshka |
|
results: |
|
- task: |
|
type: information-retrieval |
|
name: Information Retrieval |
|
dataset: |
|
name: dim 768 |
|
type: dim_768 |
|
metrics: |
|
- type: cosine_accuracy@1 |
|
value: 0.5120967741935484 |
|
name: Cosine Accuracy@1 |
|
- type: cosine_accuracy@3 |
|
value: 0.8266129032258065 |
|
name: Cosine Accuracy@3 |
|
- type: cosine_accuracy@5 |
|
value: 0.9233870967741935 |
|
name: Cosine Accuracy@5 |
|
- type: cosine_accuracy@10 |
|
value: 0.9637096774193549 |
|
name: Cosine Accuracy@10 |
|
- type: cosine_precision@1 |
|
value: 0.5120967741935484 |
|
name: Cosine Precision@1 |
|
- type: cosine_precision@3 |
|
value: 0.2755376344086021 |
|
name: Cosine Precision@3 |
|
- type: cosine_precision@5 |
|
value: 0.18467741935483872 |
|
name: Cosine Precision@5 |
|
- type: cosine_precision@10 |
|
value: 0.09637096774193549 |
|
name: Cosine Precision@10 |
|
- type: cosine_recall@1 |
|
value: 0.5120967741935484 |
|
name: Cosine Recall@1 |
|
- type: cosine_recall@3 |
|
value: 0.8266129032258065 |
|
name: Cosine Recall@3 |
|
- type: cosine_recall@5 |
|
value: 0.9233870967741935 |
|
name: Cosine Recall@5 |
|
- type: cosine_recall@10 |
|
value: 0.9637096774193549 |
|
name: Cosine Recall@10 |
|
- type: cosine_ndcg@10 |
|
value: 0.7538879073840729 |
|
name: Cosine Ndcg@10 |
|
- type: cosine_mrr@10 |
|
value: 0.6844038018433181 |
|
name: Cosine Mrr@10 |
|
- type: cosine_map@100 |
|
value: 0.6858592666542238 |
|
name: Cosine Map@100 |
|
- task: |
|
type: information-retrieval |
|
name: Information Retrieval |
|
dataset: |
|
name: dim 512 |
|
type: dim_512 |
|
metrics: |
|
- type: cosine_accuracy@1 |
|
value: 0.532258064516129 |
|
name: Cosine Accuracy@1 |
|
- type: cosine_accuracy@3 |
|
value: 0.8225806451612904 |
|
name: Cosine Accuracy@3 |
|
- type: cosine_accuracy@5 |
|
value: 0.9193548387096774 |
|
name: Cosine Accuracy@5 |
|
- type: cosine_accuracy@10 |
|
value: 0.967741935483871 |
|
name: Cosine Accuracy@10 |
|
- type: cosine_precision@1 |
|
value: 0.532258064516129 |
|
name: Cosine Precision@1 |
|
- type: cosine_precision@3 |
|
value: 0.27419354838709675 |
|
name: Cosine Precision@3 |
|
- type: cosine_precision@5 |
|
value: 0.18387096774193548 |
|
name: Cosine Precision@5 |
|
- type: cosine_precision@10 |
|
value: 0.09677419354838711 |
|
name: Cosine Precision@10 |
|
- type: cosine_recall@1 |
|
value: 0.532258064516129 |
|
name: Cosine Recall@1 |
|
- type: cosine_recall@3 |
|
value: 0.8225806451612904 |
|
name: Cosine Recall@3 |
|
- type: cosine_recall@5 |
|
value: 0.9193548387096774 |
|
name: Cosine Recall@5 |
|
- type: cosine_recall@10 |
|
value: 0.967741935483871 |
|
name: Cosine Recall@10 |
|
- type: cosine_ndcg@10 |
|
value: 0.7596718979684643 |
|
name: Cosine Ndcg@10 |
|
- type: cosine_mrr@10 |
|
value: 0.6912602406554021 |
|
name: Cosine Mrr@10 |
|
- type: cosine_map@100 |
|
value: 0.6924236134719179 |
|
name: Cosine Map@100 |
|
- task: |
|
type: information-retrieval |
|
name: Information Retrieval |
|
dataset: |
|
name: dim 256 |
|
type: dim_256 |
|
metrics: |
|
- type: cosine_accuracy@1 |
|
value: 0.5241935483870968 |
|
name: Cosine Accuracy@1 |
|
- type: cosine_accuracy@3 |
|
value: 0.8225806451612904 |
|
name: Cosine Accuracy@3 |
|
- type: cosine_accuracy@5 |
|
value: 0.9193548387096774 |
|
name: Cosine Accuracy@5 |
|
- type: cosine_accuracy@10 |
|
value: 0.9596774193548387 |
|
name: Cosine Accuracy@10 |
|
- type: cosine_precision@1 |
|
value: 0.5241935483870968 |
|
name: Cosine Precision@1 |
|
- type: cosine_precision@3 |
|
value: 0.27419354838709675 |
|
name: Cosine Precision@3 |
|
- type: cosine_precision@5 |
|
value: 0.1838709677419355 |
|
name: Cosine Precision@5 |
|
- type: cosine_precision@10 |
|
value: 0.0959677419354839 |
|
name: Cosine Precision@10 |
|
- type: cosine_recall@1 |
|
value: 0.5241935483870968 |
|
name: Cosine Recall@1 |
|
- type: cosine_recall@3 |
|
value: 0.8225806451612904 |
|
name: Cosine Recall@3 |
|
- type: cosine_recall@5 |
|
value: 0.9193548387096774 |
|
name: Cosine Recall@5 |
|
- type: cosine_recall@10 |
|
value: 0.9596774193548387 |
|
name: Cosine Recall@10 |
|
- type: cosine_ndcg@10 |
|
value: 0.7527772429981233 |
|
name: Cosine Ndcg@10 |
|
- type: cosine_mrr@10 |
|
value: 0.6846406169994881 |
|
name: Cosine Mrr@10 |
|
- type: cosine_map@100 |
|
value: 0.6862769216923534 |
|
name: Cosine Map@100 |
|
- task: |
|
type: information-retrieval |
|
name: Information Retrieval |
|
dataset: |
|
name: dim 128 |
|
type: dim_128 |
|
metrics: |
|
- type: cosine_accuracy@1 |
|
value: 0.4959677419354839 |
|
name: Cosine Accuracy@1 |
|
- type: cosine_accuracy@3 |
|
value: 0.7903225806451613 |
|
name: Cosine Accuracy@3 |
|
- type: cosine_accuracy@5 |
|
value: 0.8911290322580645 |
|
name: Cosine Accuracy@5 |
|
- type: cosine_accuracy@10 |
|
value: 0.9556451612903226 |
|
name: Cosine Accuracy@10 |
|
- type: cosine_precision@1 |
|
value: 0.4959677419354839 |
|
name: Cosine Precision@1 |
|
- type: cosine_precision@3 |
|
value: 0.26344086021505375 |
|
name: Cosine Precision@3 |
|
- type: cosine_precision@5 |
|
value: 0.17822580645161293 |
|
name: Cosine Precision@5 |
|
- type: cosine_precision@10 |
|
value: 0.09556451612903227 |
|
name: Cosine Precision@10 |
|
- type: cosine_recall@1 |
|
value: 0.4959677419354839 |
|
name: Cosine Recall@1 |
|
- type: cosine_recall@3 |
|
value: 0.7903225806451613 |
|
name: Cosine Recall@3 |
|
- type: cosine_recall@5 |
|
value: 0.8911290322580645 |
|
name: Cosine Recall@5 |
|
- type: cosine_recall@10 |
|
value: 0.9556451612903226 |
|
name: Cosine Recall@10 |
|
- type: cosine_ndcg@10 |
|
value: 0.73375586078758 |
|
name: Cosine Ndcg@10 |
|
- type: cosine_mrr@10 |
|
value: 0.6613495263696876 |
|
name: Cosine Mrr@10 |
|
- type: cosine_map@100 |
|
value: 0.6630698645438532 |
|
name: Cosine Map@100 |
|
- task: |
|
type: information-retrieval |
|
name: Information Retrieval |
|
dataset: |
|
name: dim 64 |
|
type: dim_64 |
|
metrics: |
|
- type: cosine_accuracy@1 |
|
value: 0.4475806451612903 |
|
name: Cosine Accuracy@1 |
|
- type: cosine_accuracy@3 |
|
value: 0.7661290322580645 |
|
name: Cosine Accuracy@3 |
|
- type: cosine_accuracy@5 |
|
value: 0.8790322580645161 |
|
name: Cosine Accuracy@5 |
|
- type: cosine_accuracy@10 |
|
value: 0.9475806451612904 |
|
name: Cosine Accuracy@10 |
|
- type: cosine_precision@1 |
|
value: 0.4475806451612903 |
|
name: Cosine Precision@1 |
|
- type: cosine_precision@3 |
|
value: 0.2553763440860215 |
|
name: Cosine Precision@3 |
|
- type: cosine_precision@5 |
|
value: 0.17580645161290326 |
|
name: Cosine Precision@5 |
|
- type: cosine_precision@10 |
|
value: 0.09475806451612903 |
|
name: Cosine Precision@10 |
|
- type: cosine_recall@1 |
|
value: 0.4475806451612903 |
|
name: Cosine Recall@1 |
|
- type: cosine_recall@3 |
|
value: 0.7661290322580645 |
|
name: Cosine Recall@3 |
|
- type: cosine_recall@5 |
|
value: 0.8790322580645161 |
|
name: Cosine Recall@5 |
|
- type: cosine_recall@10 |
|
value: 0.9475806451612904 |
|
name: Cosine Recall@10 |
|
- type: cosine_ndcg@10 |
|
value: 0.7052651530890945 |
|
name: Cosine Ndcg@10 |
|
- type: cosine_mrr@10 |
|
value: 0.6260768689196109 |
|
name: Cosine Mrr@10 |
|
- type: cosine_map@100 |
|
value: 0.6277483838406475 |
|
name: Cosine Map@100 |
|
--- |
|
|
|
# BGE base Financial Matryoshka |
|
|
|
This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [BAAI/bge-base-en-v1.5](https://huggingface.co./BAAI/bge-base-en-v1.5). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. |
|
|
|
## Model Details |
|
|
|
### Model Description |
|
- **Model Type:** Sentence Transformer |
|
- **Base model:** [BAAI/bge-base-en-v1.5](https://huggingface.co./BAAI/bge-base-en-v1.5) <!-- at revision a5beb1e3e68b9ab74eb54cfd186867f64f240e1a --> |
|
- **Maximum Sequence Length:** 512 tokens |
|
- **Output Dimensionality:** 768 tokens |
|
- **Similarity Function:** Cosine Similarity |
|
<!-- - **Training Dataset:** Unknown --> |
|
- **Language:** en |
|
- **License:** apache-2.0 |
|
|
|
### Model Sources |
|
|
|
- **Documentation:** [Sentence Transformers Documentation](https://sbert.net) |
|
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers) |
|
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co./models?library=sentence-transformers) |
|
|
|
### Full Model Architecture |
|
|
|
``` |
|
SentenceTransformer( |
|
(0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel |
|
(1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) |
|
(2): Normalize() |
|
) |
|
``` |
|
|
|
## Usage |
|
|
|
### Direct Usage (Sentence Transformers) |
|
|
|
First install the Sentence Transformers library: |
|
|
|
```bash |
|
pip install -U sentence-transformers |
|
``` |
|
|
|
Then you can load this model and run inference. |
|
```python |
|
from sentence_transformers import SentenceTransformer |
|
|
|
# Download from the 🤗 Hub |
|
model = SentenceTransformer("anishareddyalla/bge-base-matryoshka-aws-casestudies") |
|
# Run inference |
|
sentences = [ |
|
'Alternatively, you can run the inference via code. Here is one example written in Python, using the requests library: import requests url = "https://<YOUR_API_GATEWAY_ENDPOINT_ID>. execute-api. <YOUR_ENDPOINT_REGION>. amazonaws. com/prod/question?question=\\"What is the color of my car now?\\"&context=\\"My car used to be blue but I painted red\\"" response = requests. request("GET", url, headers=headers, data=payload) print(response. text) The code outputs a string similar to the following: \'{"score":0. 6947233080863953,"start":38,"end":41,"answer":"red"}\' If you are interested in knowing more about deploying Generative AI and large language models on AWS, check out here: Deploy Serverless Generative AI on AWS Lambda with OpenLLaMa Deploy large language models on AWS Inferentia2 using large model inference containers Clean up Inside the root directory of your repository, run the following code to clean up your resources: make destroy Conclusion In this post, we introduced how you can use Lambda to deploy your trained ML model using your preferred web application framework, such as FastAPI. We provided a detailed code repository that you can deploy, and you retain the flexibility of switching to whichever trained model artifacts you process. The performance can depend on how you implement and deploy the model. You are welcome to try it out yourself, and we’re excited to hear your feedback! About the Authors Tingyi Li is an Enterprise Solutions Architect from AWS based out in Stockholm, Sweden supporting the Nordics customers. She enjoys helping customers with the architecture, design, and development of cloud-optimized infrastructure solutions. She is specialized in AI and Machine Learning and is interested in empowering customers with intelligence in their AI/ML applications. In her spare time, she is also a part-time illustrator who writes novels and plays the piano. Demir Catovic is a Machine Learning Engineer from AWS based in Zurich, Switzerland. He engages with customers and helps them implement scalable and fully-functional ML applications. He is passionate about building and productionizing machine learning applications for customers and is always keen to explore around new trends and cutting-edge technologies in the AI/ML world. TAGS: Generative AI , Natural Language Processing Comments View Comments Resources Getting Started What\'s New Blog Topics Amazon Comprehend Amazon Kendra Amazon Lex Amazon Polly Amazon Rekognition Amazon SageMaker Amazon Textract Follow Twitter Facebook LinkedIn Twitch Email Updates.', |
|
'How can Lambda be used to deploy trained ML models using a preferred web application framework?', |
|
'How has PayEye been able to accelerate their development process and enter the production phase within a few months using AWS services, and what impact has this had on their recruitment efforts and team focus?', |
|
] |
|
embeddings = model.encode(sentences) |
|
print(embeddings.shape) |
|
# [3, 768] |
|
|
|
# Get the similarity scores for the embeddings |
|
similarities = model.similarity(embeddings, embeddings) |
|
print(similarities.shape) |
|
# [3, 3] |
|
``` |
|
|
|
<!-- |
|
### Direct Usage (Transformers) |
|
|
|
<details><summary>Click to see the direct usage in Transformers</summary> |
|
|
|
</details> |
|
--> |
|
|
|
<!-- |
|
### Downstream Usage (Sentence Transformers) |
|
|
|
You can finetune this model on your own dataset. |
|
|
|
<details><summary>Click to expand</summary> |
|
|
|
</details> |
|
--> |
|
|
|
<!-- |
|
### Out-of-Scope Use |
|
|
|
*List how the model may foreseeably be misused and address what users ought not to do with the model.* |
|
--> |
|
|
|
## Evaluation |
|
|
|
### Metrics |
|
|
|
#### Information Retrieval |
|
* Dataset: `dim_768` |
|
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) |
|
|
|
| Metric | Value | |
|
|:--------------------|:-----------| |
|
| cosine_accuracy@1 | 0.5121 | |
|
| cosine_accuracy@3 | 0.8266 | |
|
| cosine_accuracy@5 | 0.9234 | |
|
| cosine_accuracy@10 | 0.9637 | |
|
| cosine_precision@1 | 0.5121 | |
|
| cosine_precision@3 | 0.2755 | |
|
| cosine_precision@5 | 0.1847 | |
|
| cosine_precision@10 | 0.0964 | |
|
| cosine_recall@1 | 0.5121 | |
|
| cosine_recall@3 | 0.8266 | |
|
| cosine_recall@5 | 0.9234 | |
|
| cosine_recall@10 | 0.9637 | |
|
| cosine_ndcg@10 | 0.7539 | |
|
| cosine_mrr@10 | 0.6844 | |
|
| **cosine_map@100** | **0.6859** | |
|
|
|
#### Information Retrieval |
|
* Dataset: `dim_512` |
|
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) |
|
|
|
| Metric | Value | |
|
|:--------------------|:-----------| |
|
| cosine_accuracy@1 | 0.5323 | |
|
| cosine_accuracy@3 | 0.8226 | |
|
| cosine_accuracy@5 | 0.9194 | |
|
| cosine_accuracy@10 | 0.9677 | |
|
| cosine_precision@1 | 0.5323 | |
|
| cosine_precision@3 | 0.2742 | |
|
| cosine_precision@5 | 0.1839 | |
|
| cosine_precision@10 | 0.0968 | |
|
| cosine_recall@1 | 0.5323 | |
|
| cosine_recall@3 | 0.8226 | |
|
| cosine_recall@5 | 0.9194 | |
|
| cosine_recall@10 | 0.9677 | |
|
| cosine_ndcg@10 | 0.7597 | |
|
| cosine_mrr@10 | 0.6913 | |
|
| **cosine_map@100** | **0.6924** | |
|
|
|
#### Information Retrieval |
|
* Dataset: `dim_256` |
|
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) |
|
|
|
| Metric | Value | |
|
|:--------------------|:-----------| |
|
| cosine_accuracy@1 | 0.5242 | |
|
| cosine_accuracy@3 | 0.8226 | |
|
| cosine_accuracy@5 | 0.9194 | |
|
| cosine_accuracy@10 | 0.9597 | |
|
| cosine_precision@1 | 0.5242 | |
|
| cosine_precision@3 | 0.2742 | |
|
| cosine_precision@5 | 0.1839 | |
|
| cosine_precision@10 | 0.096 | |
|
| cosine_recall@1 | 0.5242 | |
|
| cosine_recall@3 | 0.8226 | |
|
| cosine_recall@5 | 0.9194 | |
|
| cosine_recall@10 | 0.9597 | |
|
| cosine_ndcg@10 | 0.7528 | |
|
| cosine_mrr@10 | 0.6846 | |
|
| **cosine_map@100** | **0.6863** | |
|
|
|
#### Information Retrieval |
|
* Dataset: `dim_128` |
|
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) |
|
|
|
| Metric | Value | |
|
|:--------------------|:-----------| |
|
| cosine_accuracy@1 | 0.496 | |
|
| cosine_accuracy@3 | 0.7903 | |
|
| cosine_accuracy@5 | 0.8911 | |
|
| cosine_accuracy@10 | 0.9556 | |
|
| cosine_precision@1 | 0.496 | |
|
| cosine_precision@3 | 0.2634 | |
|
| cosine_precision@5 | 0.1782 | |
|
| cosine_precision@10 | 0.0956 | |
|
| cosine_recall@1 | 0.496 | |
|
| cosine_recall@3 | 0.7903 | |
|
| cosine_recall@5 | 0.8911 | |
|
| cosine_recall@10 | 0.9556 | |
|
| cosine_ndcg@10 | 0.7338 | |
|
| cosine_mrr@10 | 0.6613 | |
|
| **cosine_map@100** | **0.6631** | |
|
|
|
#### Information Retrieval |
|
* Dataset: `dim_64` |
|
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) |
|
|
|
| Metric | Value | |
|
|:--------------------|:-----------| |
|
| cosine_accuracy@1 | 0.4476 | |
|
| cosine_accuracy@3 | 0.7661 | |
|
| cosine_accuracy@5 | 0.879 | |
|
| cosine_accuracy@10 | 0.9476 | |
|
| cosine_precision@1 | 0.4476 | |
|
| cosine_precision@3 | 0.2554 | |
|
| cosine_precision@5 | 0.1758 | |
|
| cosine_precision@10 | 0.0948 | |
|
| cosine_recall@1 | 0.4476 | |
|
| cosine_recall@3 | 0.7661 | |
|
| cosine_recall@5 | 0.879 | |
|
| cosine_recall@10 | 0.9476 | |
|
| cosine_ndcg@10 | 0.7053 | |
|
| cosine_mrr@10 | 0.6261 | |
|
| **cosine_map@100** | **0.6277** | |
|
|
|
<!-- |
|
## Bias, Risks and Limitations |
|
|
|
*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* |
|
--> |
|
|
|
<!-- |
|
### Recommendations |
|
|
|
*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* |
|
--> |
|
|
|
## Training Details |
|
|
|
### Training Dataset |
|
|
|
#### Unnamed Dataset |
|
|
|
|
|
* Size: 2,231 training samples |
|
* Columns: <code>positive</code> and <code>anchor</code> |
|
* Approximate statistics based on the first 1000 samples: |
|
| | positive | anchor | |
|
|:--------|:------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------| |
|
| type | string | string | |
|
| details | <ul><li>min: 3 tokens</li><li>mean: 430.06 tokens</li><li>max: 512 tokens</li></ul> | <ul><li>min: 7 tokens</li><li>mean: 33.49 tokens</li><li>max: 65 tokens</li></ul> | |
|
* Samples: |
|
| positive | anchor | |
|
|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------| |
|
| <code>TCSG is helping students enter a competitive workforce as educated cloud professionals and providing opportunities for success. TCSG built its Cloud Academy using AWS Academy, which provides higher education institutions with a free, ready-to-teach cloud computing curriculum that prepares students to pursue industry-recognized certifications and in-demand cloud jobs. TCSG launched the TCSG Cloud Academy in two forms: one as a specialization within an existing associate’s degree and the second as a stand-alone technical certificate of credit. For the technical certificate of credit, students who have existing degrees can enter the curriculum to focus on cloud computing and participate in hands-on cloud experiences using AWS services. Tiếng Việt Italiano ไทย The Technical College System of Georgia is the state government agency that supervises workforce development of more than 294,000 students across 22 technical colleges, 88 campuses, and more than 600 programs. Using the AWS curriculum and technology as the foundation for its courses, TCSG is preparing students to earn industry-recognized AWS Certifications to increase employability while improving accessibility to cloud education by offering the academy virtually and remotely. Learn more » TCSG is the state of Georgia government agency that supervises workforce development of hundreds of thousands of students across 22 technical colleges, 88 campuses, and more than 600 programs. The agency aims to run a system of technical education using the latest technology that’s accessible to all adults and corporate citizens in the state. To develop and deploy its new cloud-focused curriculum, it worked with AWS Education Programs, which helps TCSG institutions develop initiatives that align education to careers in the cloud and promote student employability, preparing diverse learners for in-demand cloud roles around the world. In 2020, the organization officially launched the TCSG Cloud Academy—a virtual program for students pursuing expertise and certifications in cloud computing—on its eCampus virtual learning system. Organizations of all sizes across all industries are transforming their businesses and delivering on their missions every day using AWS. Contact our experts and start your own AWS journey today. Português.</code> | <code>How has the use of AWS Academy by TCSG helped prepare students for pursuing industry-recognized certifications and in-demand cloud jobs in Georgia's workforce?</code> | |
|
| <code>This prompt is then provided to the LLM for generating an answer to the user question. @router. post("/rag") async def rag_handler(req: Request) -> Dict[str, Any]: # dump the received request for debugging purposes logger. info(f"req={req}") # initialize vector db and SageMaker Endpoint _init(req) # Use the vector db to find similar documents to the query # the vector db call would automatically convert the query text # into embeddings docs = _vector_db. similarity_search(req. q, k=req. max_matching_docs) logger. info(f"here are the {req. max_matching_docs} closest matching docs to the query=\"{req. q}\"") for d in docs: logger. info(f"---------") logger. info(d) logger. info(f"---------") # now that we have the matching docs, lets pack them as a context # into the prompt and ask the LLM to generate a response prompt_template = """Answer based on context:\n\n{context}\n\n{question}""" prompt = PromptTemplate( template=prompt_template, input_variables=["context", "question"] ) logger. info(f"prompt sent to llm = \"{prompt}\"") chain = load_qa_chain(llm=_sm_llm, prompt=prompt) answer = chain({"input_documents": docs, "question": req. q}, return_only_outputs=True)['output_text'] logger. info(f"answer received from llm,\nquestion: \"{req. q}\"\nanswer: \"{answer}\"") resp = {'question': req. q, 'answer': answer} if req. verbose is True: resp['docs'] = docs return resp Clean up To avoid incurring future charges, delete the resources. You can do this by deleting the CloudFormation stack as shown in the following screenshot.</code> | <code>What resources need to be deleted to avoid future charges, and how can they be deleted?</code> | |
|
| <code>append(input_1_s3_location) async_response = base_model_predictor. predict_async(input_path=input_1_s3_location) output_locations. append(async_response. output_path) if i > max_images: break This may take up to 30 minutes or more depending on how much data you have uploaded for asynchronous inference. You can visualize one of these inferences as follows: plot_response('data/single. out') Convert the asynchronous inference output to a Ground Truth input manifest In this step, we create an input manifest for a bounding box verification job on Ground Truth. We upload the Ground Truth UI template and label categories file, and create the verification job. The notebook linked to this post uses a private workforce to perform the labeling; you can change this if you’re using other types of workforces. For more details, refer to the full code in the notebook. Verify labels from the auto-labeling process in Ground Truth In this step, we complete the verification by accessing the labeling portal. For more details, refer to here. When you access the portal as a workforce member, you will be able to see the bounding boxes created by the JumpStart model and make adjustments as required. You can use this template to repeat auto-labeling with many task-specific models, potentially merge labels, and use the resulting labeled dataset in downstream tasks. Clean up In this step, we clean up by deleting the endpoint and the model created in previous steps: # Delete the SageMaker endpoint base_model_predictor. delete_model() base_model_predictor. delete_endpoint() Conclusion In this post, we walked through an auto-labeling process involving JumpStart and asynchronous inference. We used the results of the auto-labeling process to convert and visualize labeled data on a real-world dataset. You can use the solution to perform auto-labeling with many task-specific models, potentially merge labels, and use the resulting labeled dataset in downstream tasks. You can also explore using tools like the Segment Anything Model for generating segment masks as part of the auto-labeling process. In future posts in this series, we will cover the perception module and segmentation.</code> | <code>How can you visualize the inferences generated by the asynchronous inference process using the provided solution?</code> | |
|
* Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters: |
|
```json |
|
{ |
|
"loss": "MultipleNegativesRankingLoss", |
|
"matryoshka_dims": [ |
|
768, |
|
512, |
|
256, |
|
128, |
|
64 |
|
], |
|
"matryoshka_weights": [ |
|
1, |
|
1, |
|
1, |
|
1, |
|
1 |
|
], |
|
"n_dims_per_step": -1 |
|
} |
|
``` |
|
|
|
### Training Hyperparameters |
|
#### Non-Default Hyperparameters |
|
|
|
- `eval_strategy`: epoch |
|
- `per_device_train_batch_size`: 32 |
|
- `per_device_eval_batch_size`: 16 |
|
- `gradient_accumulation_steps`: 16 |
|
- `learning_rate`: 2e-05 |
|
- `num_train_epochs`: 4 |
|
- `lr_scheduler_type`: cosine |
|
- `warmup_ratio`: 0.1 |
|
- `bf16`: True |
|
- `tf32`: True |
|
- `load_best_model_at_end`: True |
|
- `optim`: adamw_torch_fused |
|
- `batch_sampler`: no_duplicates |
|
|
|
#### All Hyperparameters |
|
<details><summary>Click to expand</summary> |
|
|
|
- `overwrite_output_dir`: False |
|
- `do_predict`: False |
|
- `eval_strategy`: epoch |
|
- `prediction_loss_only`: True |
|
- `per_device_train_batch_size`: 32 |
|
- `per_device_eval_batch_size`: 16 |
|
- `per_gpu_train_batch_size`: None |
|
- `per_gpu_eval_batch_size`: None |
|
- `gradient_accumulation_steps`: 16 |
|
- `eval_accumulation_steps`: None |
|
- `learning_rate`: 2e-05 |
|
- `weight_decay`: 0.0 |
|
- `adam_beta1`: 0.9 |
|
- `adam_beta2`: 0.999 |
|
- `adam_epsilon`: 1e-08 |
|
- `max_grad_norm`: 1.0 |
|
- `num_train_epochs`: 4 |
|
- `max_steps`: -1 |
|
- `lr_scheduler_type`: cosine |
|
- `lr_scheduler_kwargs`: {} |
|
- `warmup_ratio`: 0.1 |
|
- `warmup_steps`: 0 |
|
- `log_level`: passive |
|
- `log_level_replica`: warning |
|
- `log_on_each_node`: True |
|
- `logging_nan_inf_filter`: True |
|
- `save_safetensors`: True |
|
- `save_on_each_node`: False |
|
- `save_only_model`: False |
|
- `restore_callback_states_from_checkpoint`: False |
|
- `no_cuda`: False |
|
- `use_cpu`: False |
|
- `use_mps_device`: False |
|
- `seed`: 42 |
|
- `data_seed`: None |
|
- `jit_mode_eval`: False |
|
- `use_ipex`: False |
|
- `bf16`: True |
|
- `fp16`: False |
|
- `fp16_opt_level`: O1 |
|
- `half_precision_backend`: auto |
|
- `bf16_full_eval`: False |
|
- `fp16_full_eval`: False |
|
- `tf32`: True |
|
- `local_rank`: 0 |
|
- `ddp_backend`: None |
|
- `tpu_num_cores`: None |
|
- `tpu_metrics_debug`: False |
|
- `debug`: [] |
|
- `dataloader_drop_last`: False |
|
- `dataloader_num_workers`: 0 |
|
- `dataloader_prefetch_factor`: None |
|
- `past_index`: -1 |
|
- `disable_tqdm`: False |
|
- `remove_unused_columns`: True |
|
- `label_names`: None |
|
- `load_best_model_at_end`: True |
|
- `ignore_data_skip`: False |
|
- `fsdp`: [] |
|
- `fsdp_min_num_params`: 0 |
|
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} |
|
- `fsdp_transformer_layer_cls_to_wrap`: None |
|
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} |
|
- `deepspeed`: None |
|
- `label_smoothing_factor`: 0.0 |
|
- `optim`: adamw_torch_fused |
|
- `optim_args`: None |
|
- `adafactor`: False |
|
- `group_by_length`: False |
|
- `length_column_name`: length |
|
- `ddp_find_unused_parameters`: None |
|
- `ddp_bucket_cap_mb`: None |
|
- `ddp_broadcast_buffers`: False |
|
- `dataloader_pin_memory`: True |
|
- `dataloader_persistent_workers`: False |
|
- `skip_memory_metrics`: True |
|
- `use_legacy_prediction_loop`: False |
|
- `push_to_hub`: False |
|
- `resume_from_checkpoint`: None |
|
- `hub_model_id`: None |
|
- `hub_strategy`: every_save |
|
- `hub_private_repo`: False |
|
- `hub_always_push`: False |
|
- `gradient_checkpointing`: False |
|
- `gradient_checkpointing_kwargs`: None |
|
- `include_inputs_for_metrics`: False |
|
- `eval_do_concat_batches`: True |
|
- `fp16_backend`: auto |
|
- `push_to_hub_model_id`: None |
|
- `push_to_hub_organization`: None |
|
- `mp_parameters`: |
|
- `auto_find_batch_size`: False |
|
- `full_determinism`: False |
|
- `torchdynamo`: None |
|
- `ray_scope`: last |
|
- `ddp_timeout`: 1800 |
|
- `torch_compile`: False |
|
- `torch_compile_backend`: None |
|
- `torch_compile_mode`: None |
|
- `dispatch_batches`: None |
|
- `split_batches`: None |
|
- `include_tokens_per_second`: False |
|
- `include_num_input_tokens_seen`: False |
|
- `neftune_noise_alpha`: None |
|
- `optim_target_modules`: None |
|
- `batch_eval_metrics`: False |
|
- `eval_on_start`: False |
|
- `batch_sampler`: no_duplicates |
|
- `multi_dataset_batch_sampler`: proportional |
|
|
|
</details> |
|
|
|
### Training Logs |
|
| Epoch | Step | Training Loss | dim_128_cosine_map@100 | dim_256_cosine_map@100 | dim_512_cosine_map@100 | dim_64_cosine_map@100 | dim_768_cosine_map@100 | |
|
|:----------:|:-----:|:-------------:|:----------------------:|:----------------------:|:----------------------:|:---------------------:|:----------------------:| |
|
| 0.9143 | 4 | - | 0.6663 | 0.6851 | 0.7027 | 0.6120 | 0.6998 | |
|
| **1.8286** | **8** | **-** | **0.6758** | **0.6822** | **0.6966** | **0.6311** | **0.6941** | |
|
| 2.2857 | 10 | 1.883 | - | - | - | - | - | |
|
| 2.9714 | 13 | - | 0.6631 | 0.6881 | 0.6904 | 0.6245 | 0.6873 | |
|
| 3.6571 | 16 | - | 0.6631 | 0.6863 | 0.6924 | 0.6277 | 0.6859 | |
|
|
|
* The bold row denotes the saved checkpoint. |
|
|
|
### Framework Versions |
|
- Python: 3.10.12 |
|
- Sentence Transformers: 3.0.1 |
|
- Transformers: 4.42.4 |
|
- PyTorch: 2.3.1+cu121 |
|
- Accelerate: 0.32.1 |
|
- Datasets: 2.20.0 |
|
- Tokenizers: 0.19.1 |
|
|
|
## Citation |
|
|
|
### BibTeX |
|
|
|
#### Sentence Transformers |
|
```bibtex |
|
@inproceedings{reimers-2019-sentence-bert, |
|
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", |
|
author = "Reimers, Nils and Gurevych, Iryna", |
|
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", |
|
month = "11", |
|
year = "2019", |
|
publisher = "Association for Computational Linguistics", |
|
url = "https://arxiv.org/abs/1908.10084", |
|
} |
|
``` |
|
|
|
#### MatryoshkaLoss |
|
```bibtex |
|
@misc{kusupati2024matryoshka, |
|
title={Matryoshka Representation Learning}, |
|
author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi}, |
|
year={2024}, |
|
eprint={2205.13147}, |
|
archivePrefix={arXiv}, |
|
primaryClass={cs.LG} |
|
} |
|
``` |
|
|
|
#### MultipleNegativesRankingLoss |
|
```bibtex |
|
@misc{henderson2017efficient, |
|
title={Efficient Natural Language Response Suggestion for Smart Reply}, |
|
author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil}, |
|
year={2017}, |
|
eprint={1705.00652}, |
|
archivePrefix={arXiv}, |
|
primaryClass={cs.CL} |
|
} |
|
``` |
|
|
|
<!-- |
|
## Glossary |
|
|
|
*Clearly define terms in order to be accessible across audiences.* |
|
--> |
|
|
|
<!-- |
|
## Model Card Authors |
|
|
|
*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.* |
|
--> |
|
|
|
<!-- |
|
## Model Card Contact |
|
|
|
*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.* |
|
--> |
