README.md

metadata

base_model: BAAI/bge-base-en-v1.5
datasets: []
language:
  - en
library_name: sentence-transformers
license: apache-2.0
metrics:
  - cosine_accuracy@1
  - cosine_accuracy@3
  - cosine_accuracy@5
  - cosine_accuracy@10
  - cosine_precision@1
  - cosine_precision@3
  - cosine_precision@5
  - cosine_precision@10
  - cosine_recall@1
  - cosine_recall@3
  - cosine_recall@5
  - cosine_recall@10
  - cosine_ndcg@10
  - cosine_mrr@10
  - cosine_map@100
pipeline_tag: sentence-similarity
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:2231
  - loss:MatryoshkaLoss
  - loss:MultipleNegativesRankingLoss
widget:
  - source_sentence: >-
      The fact that no customer noticed this major migration to Amazon S3
      Glacier Instant Retrieval was a big win for us. It was a seamless
      experience for end users, and we had no production issues during the
      entire migration. ” Contact Sales Greater than 99. 99% Outcome | Gaining
      Insights on AWS to Prioritize Business Needs 한국어 Snap migrated more than 2
      exabytes of data—roughly equivalent to 1. 5 trillion media
      files—seamlessly to Amazon S3 Glacier Instant Retrieval from Amazon S3
      Standard-IA. “The fact that no customer noticed this major migration to
      Amazon S3 Glacier Instant Retrieval was a big win for us,” says Manoharan.
      “It was a seamless experience for Snapchatters, and we had no production
      issues during the entire migration. ” As a result of the migration, the
      company saved tens of millions of dollars on storage. Snap has configured
      Amazon S3 in 20 AWS Regions around the world so that customers anywhere
      can retrieve data in milliseconds. The AWS Global Infrastructure is the
      most secure, extensive, and reliable Global Cloud Infrastructure for a
      business’s applications. The global reach of AWS lets Snap store media
      closer to the place where Snapchatters are creating it for optimal
      performance. Snap is also able to deliver content efficiently using Amazon
      CloudFront, a content delivery network service built for high performance,
      security, and availability. “We’ve been able to off-load all of the
      regionalization work and costs to AWS so that we can focus on developing
      new features,” says Manoharan. As a result, Snapchat continues to meet its
      quarterly cost-optimization goals. Overview | Opportunity | Solution |
      Outcome | AWS Services Used 2 exabytes Amazon Simple Storage Service
      (Amazon S3) is an object storage service offering industry-leading
      scalability, data availability, security, and performance. … In 2016, Snap
      migrated its data to AWS. “We chose to migrate to AWS because of its
      global reach, excellent performance, and competitive pricing that, in
      turn, gave us the ability to reinvest in our business,” says Vijay
      Manoharan, manager of the media delivery platform team at Snap. Amazon S3
      Glacier Instant Retrieval is an archive storage class that delivers the
      lowest-cost storage for long-lived data that is rarely accessed and
      requires retrieval in milliseconds. AWS Services Used In 2017, Snap
      migrated one of the app’s most central features—Snapchat Stories—to Amazon
      DynamoDB, a fully managed, serverless, NoSQL database designed to run
      high-performance applications at virtually any scale. Using Amazon
      DynamoDB, the company experienced greater than 99.
    sentences:
      - >-
        How did Snap save tens of millions of dollars on storage as a result of
        migrating to Amazon S3 Glacier Instant Retrieval from Amazon S3
        Standard-IA?
      - >-
        How has Panasonic Avionics Corporation leveraged Amazon Aurora
        MySQL-Compatible Edition and other AWS services to improve the
        reliability and scalability of its databases for in-flight entertainment
        and communications systems?
      - >-
        How does Ground Truth Plus ensure the quality of image and video
        captions generated by human annotators?
  - source_sentence: >-
      ” 中文 (繁體) Bahasa Indonesia Contact Sales Ρусский Customer Stories /
      Software & Internet عربي 中文 (简体) Organizations of all sizes across all
      industries are transforming their businesses and delivering on their
      missions every day using AWS. Contact our experts and start your own AWS
      journey today. Outcome | Expanding Intelligent Features of Virtual Care
      Amazon Transcribe is an automatic speech recognition service that makes it
      easy to add speech to text capabilities to any application. Learn more »
      Learn more » It is critical that video visits are secure, responsive, and
      reliable. Using AWS helps us provide all this in a performant and scalable
      way. " Overview With the Amazon Chime SDK, builders can easily add
      real-time voice, video, and messaging powered by machine learning into
      their applications. Get Started Beyond traditional use cases, Salesforce
      is adding capabilities in medication-therapy management, connectivity for
      care coordinators, and other approaches for patient engagement. The
      company is developing a new feature that will expand its support of
      Virtual Care sessions to multiple participants, instead of just clinician
      and patient. This will facilitate care-team coordination with multiple
      parties in a single meeting. Using AWS, Salesforce circumvented the heavy
      lifting that would have been required to build and maintain a
      video-calling solution from scratch. Patients self-schedule virtual
      appointments, coordinate previsit activities, and conduct virtual visits
      in a HIPAA-compliant environment. A patient’s appointment request gets
      routed to Amazon Chime SDK. Clinicians then review a patient’s intake form
      and correlate the patient to a Virtual Care session using Amazon Chime SDK
      messaging, which connects providers and patients with secure, scalable
      messaging in their web and mobile applications. The Amazon Chime SDK
      control plane sends event notifications through a default event bus to
      Amazon EventBridge, a serverless event bus that helps organizations
      receive, filter, transform, route, and deliver events. Healthcare
      professionals deliver care over the internet in near real time, which has
      significantly reduced no-shows for appointments. “Using Amazon Chime SDK,
      we don’t have to worry about the mechanics of the video call,” Daftari
      says. “We can focus on features and functions that help differentiate our
      product in the marketplace, while also significantly improving our speed
      to launch. ” Salesforce further supports accessibility through embedding
      closed-captioning of video calls using Amazon Chime SDK live
      transcription. Amazon Chime SDK sends live audio streams to Amazon
      Transcribe, which automatically converts speech to text. Salesforce Health
      Cloud customers can use the live transcription capability to display
      subtitles, create meeting transcripts, or analyze content.
    sentences:
      - >-
        How did DB Energie use Amazon SageMaker and AWS to enhance the
        sustainability and reliability of its power grid operations?
      - >-
        How did Provectus assist Earth.com in enhancing the AI-powered image
        recognition capabilities of EarthSnap and reducing engineering heavy
        lifting through the implementation of end-to-end ML pipelines and
        managed MLOps platform?
      - >-
        How does Salesforce use AWS services such as Amazon Chime SDK and Amazon
        Transcribe to enhance their Virtual Care sessions for healthcare
        professionals and patients?
  - source_sentence: >-
      It’s been a great success. ” Overview 93% Validate technical skills and
      cloud expertise to grow your career and business. Learn more » Amazon Web
      Services (AWS) Education Programs collaborate with education institutions
      and the public sector to provide access for individuals to develop cloud
      computing and digital skills. To help graduates boost their employability,
      Staffordshire University worked with the AWS team to introduce cloud
      computing skills training and add cloud courses to its credit-bearing
      computer science modules. Staffordshire University offers courses through
      AWS Academy, which empowers higher education institutions to prepare
      students for industry-recognized certifications and careers. Since the
      university added AWS Academy courses to its curriculum in 2017, several
      hundred students have participated. Of those students, 93 percent have
      achieved employment within 6 months of graduation. Empowered students
      Türkçe Solution | Learning by Doing Using AWS Learner Labs English With
      AWS Academy, our students love that they’re not just taking theory
      lessons. They get to work in actual environments with real AWS tools. ”
      Next up, Staffordshire University is expanding on the success of its cloud
      courses by launching additional programs of study developed in
      collaboration with the AWS team. Staffordshire University and the AWS team
      designed these programs by "Working Backwards" — an Amazon process that
      encourages companies to brainstorm solutions by using a customer challenge
      as the starting point — from the cloud skills employers are currently
      seeking in the United Kingdom and across the global labor market. One of
      these programs, which launches in September 2022, is a cloud computing
      course that features both cloud computing and cybersecurity modules and
      will offer students more opportunities to discover what’s possible with
      the AWS Cloud. “What we want to encourage is for students to play with AWS
      services as well as build confidence with the tools,” says Dr. Champion.
      to learn remotely using any hardware and earn AWS Certifications
      Staffordshire University added cloud computing skills training to its
      curriculum using AWS Education Programs, helping 93 percent of
      participants find employment within 6 months of graduation. covering cloud
      skills AWS Certification during the AWS Educate University Challenge
      Deutsch of graduates find jobs within 6 months Tiếng Việt Italiano ไทย
      Outcome | Developing New Cloud Coursework About Staffordshire University
      Staffordshire University is a public research university in Staffordshire,
      England. Founded in 1914, the university serves over 15,000 students
      across three schools and four campuses. The United Kingdom has experienced
      a technology boom in recent years, with technology funding tripling in the
      first 6 months of 2021 compared to the same period in 2020. In particular,
      employers need professionals with cloud computing skills ranging from
      cloud development to machine learning and data analytics. To meet demand,
      Staffordshire University offers students their choice of six AWS courses
      covering these key skills and more.
    sentences:
      - >-
        How has the collaboration between Staffordshire University and the AWS
        team impacted the employability of graduates in the field of cloud
        computing?
      - >-
        How can the confidence scores be used to verify the accuracy of
        sentiment assignments in the sentiment_results_final table, especially
        for any dubious sentiment assignments?
      - >-
        How did migrating to AWS help Travian Games improve the stability and
        reliability of their game servers, and what impact did this have on
        their players' experience?
  - source_sentence: >-
      Contact our experts and start your own AWS journey today. customer and
      agent experience 2022 Overview WaFd Bank Transforms Contact Centers Using
      Conversational AI on AWS Customer Stories / Financial Services WaFd uses a
      data lake on AWS to store and analyze data from phone and chatbot
      conversations. “We’re getting incredible data from AWS through the
      conversational logs,” says Hubbard. “That has given us insights into what
      our customers are asking for so that we can add more self-service
      functionality. ” The data also gives WaFd more insight into call volumes,
      so the call center can better manage staff schedules. Opportunity | Using
      Amazon Lex to Implement an AI-Powered Contact Center Solution Türkçe
      English WaFd is a US retail and commercial bank with over 200 branches in
      eight states. In 2019, WaFd founded subsidiary Pike Street Labs, a fintech
      startup, to drive client-facing digital innovation for the bank. “Banks
      need to meet customers’ digital expectations,” says Dustin Hubbard, chief
      technology officer at WaFd Bank and Pike Street Labs. “Every year,
      customers expect more innovation because that’s what they see from new
      entrants or in other markets. ” Pike Street Labs redesigned WaFd’s online
      banking solution to provide personalized customer experiences and began
      tackling the bank’s customer care center. The company’s previous contact
      center solution used dated technology with limited features spread across
      disparate systems. This led to long wait times for customers and
      frustration for agents, who had to answer incoming calls without prior
      knowledge of what the customer needed. Agents also bore the burden of
      identifying fraudulent calls. WaFd needed a solution to improve both the
      customer and agent experiences. Previously, WaFd used two different
      systems in its customer care center to manage its voice and chat-based
      customer interactions, with no way for one system to recognize that an
      agent was busy on the other. Chat messages remained unanswered because
      agents would forget to sign in to the chat system. The company implemented
      chatbots and voice bots powered by Amazon Lex. Now, the call and chat
      systems are interoperable, and chats can be escalated to agent assisted
      calls when needed. When a call gets passed to an agent, the system also
      passes the full chat record and an analysis of the customer’s tone so that
      the agent is prepared to address the client’s needs and be empathetic
      toward the caller’s sentiment. WaFd worked with the AWS and Talkdesk teams
      to create and launch its new contact center solution in July 2022.
    sentences:
      - >-
        How did Yellow Class optimize its video files and improve performance
        using AWS services such as AWS Elemental MediaConvert?
      - >-
        How has FanDuel ensured the redundancy and reliability of its live video
        streams through the use of AWS Elemental MediaConnect and AWS Elemental
        MediaLive?
      - >-
        How did WaFd Bank use data from phone and chatbot conversations stored
        in a data lake on AWS to improve self-service functionality and better
        manage call center staff schedules?
  - source_sentence: >-
      Alternatively, you can run the inference via code. Here is one example
      written in Python, using the requests library: import requests url =
      "https://<YOUR_API_GATEWAY_ENDPOINT_ID>. execute-api.
      <YOUR_ENDPOINT_REGION>. amazonaws. com/prod/question?question=\"What is
      the color of my car now?\"&context=\"My car used to be blue but I painted
      red\"" response = requests. request("GET", url, headers=headers,
      data=payload) print(response. text) The code outputs a string similar to
      the following: '{"score":0.
      6947233080863953,"start":38,"end":41,"answer":"red"}' If you are
      interested in knowing more about deploying Generative AI and large
      language models on AWS, check out here: Deploy Serverless Generative AI on
      AWS Lambda with OpenLLaMa Deploy large language models on AWS Inferentia2
      using large model inference containers Clean up Inside the root directory
      of your repository, run the following code to clean up your resources:
      make destroy Conclusion In this post, we introduced how you can use Lambda
      to deploy your trained ML model using your preferred web application
      framework, such as FastAPI. We provided a detailed code repository that
      you can deploy, and you retain the flexibility of switching to whichever
      trained model artifacts you process. The performance can depend on how you
      implement and deploy the model. You are welcome to try it out yourself,
      and we’re excited to hear your feedback! About the Authors Tingyi Li is an
      Enterprise Solutions Architect from AWS based out in Stockholm, Sweden
      supporting the Nordics customers. She enjoys helping customers with the
      architecture, design, and development of cloud-optimized infrastructure
      solutions. She is specialized in AI and Machine Learning and is interested
      in empowering customers with intelligence in their AI/ML applications. In
      her spare time, she is also a part-time illustrator who writes novels and
      plays the piano. Demir Catovic is a Machine Learning Engineer from AWS
      based in Zurich, Switzerland. He engages with customers and helps them
      implement scalable and fully-functional ML applications. He is passionate
      about building and productionizing machine learning applications for
      customers and is always keen to explore around new trends and cutting-edge
      technologies in the AI/ML world. TAGS: Generative AI , Natural Language
      Processing Comments View Comments Resources Getting Started What's New
      Blog Topics Amazon Comprehend Amazon Kendra Amazon Lex Amazon Polly Amazon
      Rekognition Amazon SageMaker Amazon Textract Follow Twitter Facebook
      LinkedIn Twitch Email Updates.
    sentences:
      - >-
        How did ALTBalaji use AWS Elemental MediaLive to handle a tenfold
        increase in viewership during the live streaming of Lock Upp, and what
        insights did they gain from this experience?
      - >-
        How has PayEye been able to accelerate their development process and
        enter the production phase within a few months using AWS services, and
        what impact has this had on their recruitment efforts and team focus?
      - >-
        How can Lambda be used to deploy trained ML models using a preferred web
        application framework?
model-index:
  - name: BGE base Financial Matryoshka
    results:
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 768
          type: dim_768
        metrics:
          - type: cosine_accuracy@1
            value: 0.5120967741935484
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.8266129032258065
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.9233870967741935
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.9637096774193549
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.5120967741935484
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.2755376344086021
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.18467741935483872
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.09637096774193549
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.5120967741935484
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.8266129032258065
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.9233870967741935
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.9637096774193549
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.7538879073840729
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.6844038018433181
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.6858592666542238
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 512
          type: dim_512
        metrics:
          - type: cosine_accuracy@1
            value: 0.532258064516129
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.8225806451612904
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.9193548387096774
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.967741935483871
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.532258064516129
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.27419354838709675
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.18387096774193548
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.09677419354838711
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.532258064516129
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.8225806451612904
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.9193548387096774
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.967741935483871
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.7596718979684643
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.6912602406554021
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.6924236134719179
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 256
          type: dim_256
        metrics:
          - type: cosine_accuracy@1
            value: 0.5241935483870968
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.8225806451612904
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.9193548387096774
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.9596774193548387
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.5241935483870968
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.27419354838709675
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.1838709677419355
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.0959677419354839
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.5241935483870968
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.8225806451612904
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.9193548387096774
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.9596774193548387
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.7527772429981233
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.6846406169994881
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.6862769216923534
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 128
          type: dim_128
        metrics:
          - type: cosine_accuracy@1
            value: 0.4959677419354839
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.7903225806451613
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.8911290322580645
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.9556451612903226
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.4959677419354839
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.26344086021505375
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.17822580645161293
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.09556451612903227
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.4959677419354839
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.7903225806451613
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.8911290322580645
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.9556451612903226
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.73375586078758
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.6613495263696876
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.6630698645438532
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 64
          type: dim_64
        metrics:
          - type: cosine_accuracy@1
            value: 0.4475806451612903
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.7661290322580645
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.8790322580645161
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.9475806451612904
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.4475806451612903
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.2553763440860215
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.17580645161290326
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.09475806451612903
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.4475806451612903
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.7661290322580645
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.8790322580645161
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.9475806451612904
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.7052651530890945
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.6260768689196109
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.6277483838406475
            name: Cosine Map@100

BGE base Financial Matryoshka

This is a sentence-transformers model finetuned from BAAI/bge-base-en-v1.5. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: BAAI/bge-base-en-v1.5
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 tokens
• Similarity Function: Cosine Similarity
• Language: en
• License: apache-2.0

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("anishareddyalla/bge-base-matryoshka-aws-casestudies")
# Run inference
sentences = [
    'Alternatively, you can run the inference via code. Here is one example written in Python, using the requests library: import requests url = "https://<YOUR_API_GATEWAY_ENDPOINT_ID>. execute-api. <YOUR_ENDPOINT_REGION>. amazonaws. com/prod/question?question=\\"What is the color of my car now?\\"&context=\\"My car used to be blue but I painted red\\"" response = requests. request("GET", url, headers=headers, data=payload) print(response. text) The code outputs a string similar to the following: \'{"score":0. 6947233080863953,"start":38,"end":41,"answer":"red"}\' If you are interested in knowing more about deploying Generative AI and large language models on AWS, check out here: Deploy Serverless Generative AI on AWS Lambda with OpenLLaMa Deploy large language models on AWS Inferentia2 using large model inference containers Clean up Inside the root directory of your repository, run the following code to clean up your resources: make destroy Conclusion In this post, we introduced how you can use Lambda to deploy your trained ML model using your preferred web application framework, such as FastAPI. We provided a detailed code repository that you can deploy, and you retain the flexibility of switching to whichever trained model artifacts you process. The performance can depend on how you implement and deploy the model. You are welcome to try it out yourself, and we’re excited to hear your feedback! About the Authors Tingyi Li is an Enterprise Solutions Architect from AWS based out in Stockholm, Sweden supporting the Nordics customers. She enjoys helping customers with the architecture, design, and development of cloud-optimized infrastructure solutions. She is specialized in AI and Machine Learning and is interested in empowering customers with intelligence in their AI/ML applications. In her spare time, she is also a part-time illustrator who writes novels and plays the piano. Demir Catovic is a Machine Learning Engineer from AWS based in Zurich, Switzerland. He engages with customers and helps them implement scalable and fully-functional ML applications. He is passionate about building and productionizing machine learning applications for customers and is always keen to explore around new trends and cutting-edge technologies in the AI/ML world. TAGS: Generative AI , Natural Language Processing Comments View Comments Resources Getting Started What\'s New Blog Topics Amazon Comprehend Amazon Kendra Amazon Lex Amazon Polly Amazon Rekognition Amazon SageMaker Amazon Textract Follow Twitter Facebook LinkedIn Twitch Email Updates.',
    'How can Lambda be used to deploy trained ML models using a preferred web application framework?',
    'How has PayEye been able to accelerate their development process and enter the production phase within a few months using AWS services, and what impact has this had on their recruitment efforts and team focus?',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

• Dataset: dim_768
• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.5121
cosine_accuracy@3	0.8266
cosine_accuracy@5	0.9234
cosine_accuracy@10	0.9637
cosine_precision@1	0.5121
cosine_precision@3	0.2755
cosine_precision@5	0.1847
cosine_precision@10	0.0964
cosine_recall@1	0.5121
cosine_recall@3	0.8266
cosine_recall@5	0.9234
cosine_recall@10	0.9637
cosine_ndcg@10	0.7539
cosine_mrr@10	0.6844
cosine_map@100	0.6859

Information Retrieval

• Dataset: dim_512
• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.5323
cosine_accuracy@3	0.8226
cosine_accuracy@5	0.9194
cosine_accuracy@10	0.9677
cosine_precision@1	0.5323
cosine_precision@3	0.2742
cosine_precision@5	0.1839
cosine_precision@10	0.0968
cosine_recall@1	0.5323
cosine_recall@3	0.8226
cosine_recall@5	0.9194
cosine_recall@10	0.9677
cosine_ndcg@10	0.7597
cosine_mrr@10	0.6913
cosine_map@100	0.6924

Information Retrieval

• Dataset: dim_256
• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.5242
cosine_accuracy@3	0.8226
cosine_accuracy@5	0.9194
cosine_accuracy@10	0.9597
cosine_precision@1	0.5242
cosine_precision@3	0.2742
cosine_precision@5	0.1839
cosine_precision@10	0.096
cosine_recall@1	0.5242
cosine_recall@3	0.8226
cosine_recall@5	0.9194
cosine_recall@10	0.9597
cosine_ndcg@10	0.7528
cosine_mrr@10	0.6846
cosine_map@100	0.6863

Information Retrieval

• Dataset: dim_128
• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.496
cosine_accuracy@3	0.7903
cosine_accuracy@5	0.8911
cosine_accuracy@10	0.9556
cosine_precision@1	0.496
cosine_precision@3	0.2634
cosine_precision@5	0.1782
cosine_precision@10	0.0956
cosine_recall@1	0.496
cosine_recall@3	0.7903
cosine_recall@5	0.8911
cosine_recall@10	0.9556
cosine_ndcg@10	0.7338
cosine_mrr@10	0.6613
cosine_map@100	0.6631

Information Retrieval

• Dataset: dim_64
• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.4476
cosine_accuracy@3	0.7661
cosine_accuracy@5	0.879
cosine_accuracy@10	0.9476
cosine_precision@1	0.4476
cosine_precision@3	0.2554
cosine_precision@5	0.1758
cosine_precision@10	0.0948
cosine_recall@1	0.4476
cosine_recall@3	0.7661
cosine_recall@5	0.879
cosine_recall@10	0.9476
cosine_ndcg@10	0.7053
cosine_mrr@10	0.6261
cosine_map@100	0.6277

Training Details

Training Dataset

Unnamed Dataset

• Size: 2,231 training samples
• Columns: positive and anchor
• Approximate statistics based on the first 1000 samples:

  	positive	anchor
  type	string	string
  details	min: 3 tokens mean: 430.06 tokens max: 512 tokens	min: 7 tokens mean: 33.49 tokens max: 65 tokens

• Samples:

  positive	anchor
  TCSG is helping students enter a competitive workforce as educated cloud professionals and providing opportunities for success. TCSG built its Cloud Academy using AWS Academy, which provides higher education institutions with a free, ready-to-teach cloud computing curriculum that prepares students to pursue industry-recognized certifications and in-demand cloud jobs. TCSG launched the TCSG Cloud Academy in two forms: one as a specialization within an existing associate’s degree and the second as a stand-alone technical certificate of credit. For the technical certificate of credit, students who have existing degrees can enter the curriculum to focus on cloud computing and participate in hands-on cloud experiences using AWS services. Tiếng Việt Italiano ไทย The Technical College System of Georgia is the state government agency that supervises workforce development of more than 294,000 students across 22 technical colleges, 88 campuses, and more than 600 programs. Using the AWS curriculum and technology as the foundation for its courses, TCSG is preparing students to earn industry-recognized AWS Certifications to increase employability while improving accessibility to cloud education by offering the academy virtually and remotely. Learn more » TCSG is the state of Georgia government agency that supervises workforce development of hundreds of thousands of students across 22 technical colleges, 88 campuses, and more than 600 programs. The agency aims to run a system of technical education using the latest technology that’s accessible to all adults and corporate citizens in the state. To develop and deploy its new cloud-focused curriculum, it worked with AWS Education Programs, which helps TCSG institutions develop initiatives that align education to careers in the cloud and promote student employability, preparing diverse learners for in-demand cloud roles around the world. In 2020, the organization officially launched the TCSG Cloud Academy—a virtual program for students pursuing expertise and certifications in cloud computing—on its eCampus virtual learning system. Organizations of all sizes across all industries are transforming their businesses and delivering on their missions every day using AWS. Contact our experts and start your own AWS journey today. Português.	How has the use of AWS Academy by TCSG helped prepare students for pursuing industry-recognized certifications and in-demand cloud jobs in Georgia's workforce?
  This prompt is then provided to the LLM for generating an answer to the user question. @router. post("/rag") async def rag_handler(req: Request) -> Dict[str, Any]: # dump the received request for debugging purposes logger. info(f"req={req}") # initialize vector db and SageMaker Endpoint _init(req) # Use the vector db to find similar documents to the query # the vector db call would automatically convert the query text # into embeddings docs = _vector_db. similarity_search(req. q, k=req. max_matching_docs) logger. info(f"here are the {req. max_matching_docs} closest matching docs to the query="{req. q}"") for d in docs: logger. info(f"---------") logger. info(d) logger. info(f"---------") # now that we have the matching docs, lets pack them as a context # into the prompt and ask the LLM to generate a response prompt_template = """Answer based on context:\n\n{context}\n\n{question}""" prompt = PromptTemplate( template=prompt_template, input_variables=["context", "question"] ) logger. info(f"prompt sent to llm = "{prompt}"") chain = load_qa_chain(llm=_sm_llm, prompt=prompt) answer = chain({"input_documents": docs, "question": req. q}, return_only_outputs=True)['output_text'] logger. info(f"answer received from llm,\nquestion: "{req. q}"\nanswer: "{answer}"") resp = {'question': req. q, 'answer': answer} if req. verbose is True: resp['docs'] = docs return resp Clean up To avoid incurring future charges, delete the resources. You can do this by deleting the CloudFormation stack as shown in the following screenshot.	What resources need to be deleted to avoid future charges, and how can they be deleted?
  append(input_1_s3_location) async_response = base_model_predictor. predict_async(input_path=input_1_s3_location) output_locations. append(async_response. output_path) if i > max_images: break This may take up to 30 minutes or more depending on how much data you have uploaded for asynchronous inference. You can visualize one of these inferences as follows: plot_response('data/single. out') Convert the asynchronous inference output to a Ground Truth input manifest In this step, we create an input manifest for a bounding box verification job on Ground Truth. We upload the Ground Truth UI template and label categories file, and create the verification job. The notebook linked to this post uses a private workforce to perform the labeling; you can change this if you’re using other types of workforces. For more details, refer to the full code in the notebook. Verify labels from the auto-labeling process in Ground Truth In this step, we complete the verification by accessing the labeling portal. For more details, refer to here. When you access the portal as a workforce member, you will be able to see the bounding boxes created by the JumpStart model and make adjustments as required. You can use this template to repeat auto-labeling with many task-specific models, potentially merge labels, and use the resulting labeled dataset in downstream tasks. Clean up In this step, we clean up by deleting the endpoint and the model created in previous steps: # Delete the SageMaker endpoint base_model_predictor. delete_model() base_model_predictor. delete_endpoint() Conclusion In this post, we walked through an auto-labeling process involving JumpStart and asynchronous inference. We used the results of the auto-labeling process to convert and visualize labeled data on a real-world dataset. You can use the solution to perform auto-labeling with many task-specific models, potentially merge labels, and use the resulting labeled dataset in downstream tasks. You can also explore using tools like the Segment Anything Model for generating segment masks as part of the auto-labeling process. In future posts in this series, we will cover the perception module and segmentation.	How can you visualize the inferences generated by the asynchronous inference process using the provided solution?

• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          768,
          512,
          256,
          128,
          64
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: epoch
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 16
• gradient_accumulation_steps: 16
• learning_rate: 2e-05
• num_train_epochs: 4
• lr_scheduler_type: cosine
• warmup_ratio: 0.1
• bf16: True
• tf32: True
• load_best_model_at_end: True
• optim: adamw_torch_fused
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: epoch
• prediction_loss_only: True
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 16
• eval_accumulation_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 4
• max_steps: -1
• lr_scheduler_type: cosine
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: True
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: False
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional

Training Logs

Epoch	Step	Training Loss	dim_128_cosine_map@100	dim_256_cosine_map@100	dim_512_cosine_map@100	dim_64_cosine_map@100	dim_768_cosine_map@100
0.9143	4	-	0.6663	0.6851	0.7027	0.6120	0.6998
1.8286	8	-	0.6758	0.6822	0.6966	0.6311	0.6941
2.2857	10	1.883	-	-	-	-	-
2.9714	13	-	0.6631	0.6881	0.6904	0.6245	0.6873
3.6571	16	-	0.6631	0.6863	0.6924	0.6277	0.6859

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.10.12
• Sentence Transformers: 3.0.1
• Transformers: 4.42.4
• PyTorch: 2.3.1+cu121
• Accelerate: 0.32.1
• Datasets: 2.20.0
• Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning}, 
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply}, 
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}