metadata
base_model: sentence-transformers/all-MiniLM-L6-v2
library_name: sentence-transformers
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:274
- loss:MultipleNegativesRankingLoss
widget:
- source_sentence: >-
What are the key considerations for conducting diligence on training data
use in relation to intellectual property and privacy risks?
sentences:
- >-
APPENDIX
•
Julia Simon-Mishel, Supervising Attorney, Philadelphia Legal Assistance
•
Dr. Zachary Mahafza, Research & Data Analyst, Southern Poverty Law
Center
•
J. Khadijah Abdurahman, Tech Impact Network Research Fellow, AI Now
Institute, UCLA C2I1, and
UWA Law School
Panelists separately described the increasing scope of technology use in
providing for social welfare, including
in fraud detection, digital ID systems, and other methods focused on
improving efficiency and reducing cost.
However, various panelists individually cautioned that these systems may
reduce burden for government
agencies by increasing the burden and agency of people using and
interacting with these technologies.
Additionally, these systems can produce feedback loops and compounded
harm, collecting data from
communities and using it to reinforce inequality. Various panelists
suggested that these harms could be
mitigated by ensuring community input at the beginning of the design
process, providing ways to opt out of
these systems and use associated human-driven mechanisms instead,
ensuring timeliness of benefit payments,
and providing clear notice about the use of these systems and clear
explanations of how and what the
technologies are doing. Some panelists suggested that technology should
be used to help people receive
benefits, e.g., by pushing benefits to those in need and ensuring
automated decision-making systems are only
used to provide a positive outcome; technology shouldn't be used to take
supports away from people who need
them.
Panel 6: The Healthcare System. This event explored current and emerging
uses of technology in the
healthcare system and consumer products related to health.
Welcome:
•
Alondra Nelson, Deputy Director for Science and Society, White House
Office of Science and Technology
Policy
•
Patrick Gaspard, President and CEO, Center for American Progress
Moderator: Micky Tripathi, National Coordinator for Health Information
Technology, U.S Department of
Health and Human Services.
Panelists:
•
Mark Schneider, Health Innovation Advisor, ChristianaCare
•
Ziad Obermeyer, Blue Cross of California Distinguished Associate
Professor of Policy and Management,
University of California, Berkeley School of Public Health
•
Dorothy Roberts, George A. Weiss University Professor of Law and
Sociology and the Raymond Pace and
Sadie Tanner Mossell Alexander Professor of Civil Rights, University of
Pennsylvania
•
David Jones, A. Bernard Ackerman Professor of the Culture of Medicine,
Harvard University
•
Jamila Michener, Associate Professor of Government, Cornell University;
Co-Director, Cornell Center for
Health Equity
Panelists discussed the impact of new technologies on health
disparities; healthcare access, delivery, and
outcomes; and areas ripe for research and policymaking. Panelists
discussed the increasing importance of tech-
nology as both a vehicle to deliver healthcare and a tool to enhance the
quality of care. On the issue of
delivery, various panelists pointed to a number of concerns including
access to and expense of broadband
service, the privacy concerns associated with telehealth systems, the
expense associated with health
monitoring devices, and how this can exacerbate equity issues. On the
issue of technology enhanced care,
some panelists spoke extensively about the way in which racial biases
and the use of race in medicine
perpetuate harms and embed prior discrimination, and the importance of
ensuring that the technologies used
in medical care were accountable to the relevant stakeholders. Various
panelists emphasized the importance
of having the voices of those subjected to these technologies be heard.
59
- >-
27
MP-4.1-010
Conduct appropriate diligence on training data use to assess
intellectual property,
and privacy, risks, including to examine whether use of proprietary or
sensitive
training data is consistent with applicable laws.
Intellectual Property; Data Privacy
AI Actor Tasks: Governance and Oversight, Operation and Monitoring,
Procurement, Third-party entities
MAP 5.1: Likelihood and magnitude of each identified impact (both
potentially beneficial and harmful) based on expected use, past
uses of AI systems in similar contexts, public incident reports,
feedback from those external to the team that developed or deployed
the AI system, or other data are identified and documented.
Action ID
Suggested Action
GAI Risks
MP-5.1-001 Apply TEVV practices for content provenance (e.g., probing a
system's synthetic
data generation capabilities for potential misuse or vulnerabilities.
Information Integrity; Information
Security
MP-5.1-002
Identify potential content provenance harms of GAI, such as
misinformation or
disinformation, deepfakes, including NCII, or tampered content.
Enumerate and
rank risks based on their likelihood and potential impact, and determine
how well
provenance solutions address specific risks and/or harms.
Information Integrity; Dangerous,
Violent, or Hateful Content;
Obscene, Degrading, and/or
Abusive Content
MP-5.1-003
Consider disclosing use of GAI to end users in relevant contexts, while
considering
the objective of disclosure, the context of use, the likelihood and
magnitude of the
risk posed, the audience of the disclosure, as well as the frequency of
the
disclosures.
Human-AI Configuration
MP-5.1-004 Prioritize GAI structured public feedback processes based on
risk assessment
estimates.
Information Integrity; CBRN
Information or Capabilities;
Dangerous, Violent, or Hateful
Content; Harmful Bias and
Homogenization
MP-5.1-005 Conduct adversarial role-playing exercises, GAI red-teaming,
or chaos testing to
identify anomalous or unforeseen failure modes.
Information Security
MP-5.1-006
Profile threats and negative impacts arising from GAI systems interacting
with,
manipulating, or generating content, and outlining known and potential
vulnerabilities and the likelihood of their occurrence.
Information Security
AI Actor Tasks: AI Deployment, AI Design, AI Development, AI Impact
Assessment, Affected Individuals and Communities, End-
Users, Operation and Monitoring
- >-
18
GOVERN 3.2: Policies and procedures are in place to define and
differentiate roles and responsibilities for human-AI configurations
and oversight of AI systems.
Action ID
Suggested Action
GAI Risks
GV-3.2-001
Policies are in place to bolster oversight of GAI systems with
independent
evaluations or assessments of GAI models or systems where the type and
robustness of evaluations are proportional to the identified risks.
CBRN Information or Capabilities;
Harmful Bias and Homogenization
GV-3.2-002
Consider adjustment of organizational roles and components across
lifecycle
stages of large or complex GAI systems, including: Test and evaluation,
validation,
and red-teaming of GAI systems; GAI content moderation; GAI system
development and engineering; Increased accessibility of GAI tools,
interfaces, and
systems, Incident response and containment.
Human-AI Configuration;
Information Security; Harmful Bias
and Homogenization
GV-3.2-003
Define acceptable use policies for GAI interfaces, modalities, and
human-AI
configurations (i.e., for chatbots and decision-making tasks), including
criteria for
the kinds of queries GAI applications should refuse to respond to.
Human-AI Configuration
GV-3.2-004
Establish policies for user feedback mechanisms for GAI systems which
include
thorough instructions and any mechanisms for recourse.
Human-AI Configuration
GV-3.2-005
Engage in threat modeling to anticipate potential risks from GAI
systems.
CBRN Information or Capabilities;
Information Security
AI Actors: AI Design
GOVERN 4.1: Organizational policies and practices are in place to foster
a critical thinking and safety-first mindset in the design,
development, deployment, and uses of AI systems to minimize potential
negative impacts.
Action ID
Suggested Action
GAI Risks
GV-4.1-001
Establish policies and procedures that address continual improvement
processes
for GAI risk measurement. Address general risks associated with a lack
of
explainability and transparency in GAI systems by using ample
documentation and
techniques such as: application of gradient-based attributions,
occlusion/term
reduction, counterfactual prompts and prompt engineering, and analysis
of
embeddings; Assess and update risk measurement approaches at regular
cadences.
Confabulation
GV-4.1-002
Establish policies, procedures, and processes detailing risk measurement
in
context of use with standardized measurement protocols and structured
public
feedback exercises such as AI red-teaming or independent external
evaluations.
CBRN Information and Capability;
Value Chain and Component
Integration
- source_sentence: >-
What should individuals be able to do when encountering problems with
automated systems, according to the context provided?
sentences:
- >-
6
2.2. Confabulation
“Confabulation” refers to a phenomenon in which GAI systems generate and
confidently present
erroneous or false content in response to prompts. Confabulations also
include generated outputs that
diverge from the prompts or other input or that contradict previously
generated statements in the same
context. These phenomena are colloquially also referred to as
“hallucinations” or “fabrications.”
Confabulations can occur across GAI outputs and contexts.9,10
Confabulations are a natural result of the
way generative models are designed: they generate outputs that
approximate the statistical distribution
of their training data; for example, LLMs predict the next token or word
in a sentence or phrase. While
such statistical prediction can produce factually accurate and
consistent outputs, it can also produce
outputs that are factually inaccurate or internally inconsistent. This
dynamic is particularly relevant when
it comes to open-ended prompts for long-form responses and in domains
which require highly
contextual and/or domain expertise.
Risks from confabulations may arise when users believe false content –
often due to the confident nature
of the response – leading users to act upon or promote the false
information. This poses a challenge for
many real-world applications, such as in healthcare, where a
confabulated summary of patient
information reports could cause doctors to make incorrect diagnoses
and/or recommend the wrong
treatments. Risks of confabulated content may be especially important to
monitor when integrating GAI
into applications involving consequential decision making.
GAI outputs may also include confabulated logic or citations that
purport to justify or explain the
system’s answer, which may further mislead humans into inappropriately
trusting the system’s output.
For instance, LLMs sometimes provide logical steps for how they arrived
at an answer even when the
answer itself is incorrect. Similarly, an LLM could falsely assert that
it is human or has human traits,
potentially deceiving humans into believing they are speaking with
another human.
The extent to which humans can be deceived by LLMs, the mechanisms by
which this may occur, and the
potential risks from adversarial prompting of such behavior are emerging
areas of study. Given the wide
range of downstream impacts of GAI, it is difficult to estimate the
downstream scale and impact of
confabulations.
Trustworthy AI Characteristics: Fair with Harmful Bias Managed, Safe,
Valid and Reliable, Explainable
and Interpretable
2.3. Dangerous, Violent, or Hateful Content
GAI systems can produce content that is inciting, radicalizing, or
threatening, or that glorifies violence,
with greater ease and scale than other technologies. LLMs have been
reported to generate dangerous or
violent recommendations, and some models have generated actionable
instructions for dangerous or
9 Confabulations of falsehoods are most commonly a problem for
text-based outputs; for audio, image, or video
content, creative generation of non-factual content can be a desired
behavior.
10 For example, legal confabulations have been shown to be pervasive in
current state-of-the-art LLMs. See also,
e.g.,
- >-
SECTION TITLE
HUMAN ALTERNATIVES, CONSIDERATION, AND FALLBACK
You should be able to opt out, where appropriate, and have access to a
person who can quickly
consider and remedy problems you encounter. You should be able to opt
out from automated systems in
favor of a human alternative, where appropriate. Appropriateness should
be determined based on reasonable
expectations in a given context and with a focus on ensuring broad
accessibility and protecting the public from
especially harmful impacts. In some cases, a human or other alternative
may be required by law. You should have
access to timely human consideration and remedy by a fallback and
escalation process if an automated system
fails, it produces an error, or you would like to appeal or contest its
impacts on you. Human consideration and
fallback should be accessible, equitable, effective, maintained,
accompanied by appropriate operator training, and
should not impose an unreasonable burden on the public. Automated
systems with an intended use within sensi
tive domains, including, but not limited to, criminal justice,
employment, education, and health, should additional
ly be tailored to the purpose, provide meaningful access for oversight,
include training for any people interacting
with the system, and incorporate human consideration for adverse or
high-risk decisions. Reporting that includes
a description of these human governance processes and assessment of
their timeliness, accessibility, outcomes,
and effectiveness should be made public whenever possible.
Definitions for key terms in The Blueprint for an AI Bill of Rights can
be found in Applying the Blueprint for an AI Bill of Rights.
Accompanying analysis and tools for actualizing each principle can be
found in the Technical Companion.
7
- |-
FROM
PRINCIPLES
TO PRACTICE
A TECHINCAL COMPANION TO
THE Blueprint for an
AI BILL OF RIGHTS
12
- source_sentence: >-
How did the White House Office of Science and Technology Policy gather
input from the American public regarding algorithmic and data-driven
harms?
sentences:
- >-
23
MP-1.1-002
Determine and document the expected and acceptable GAI system context
of
use in collaboration with socio-cultural and other domain experts, by
assessing:
Assumptions and limitations; Direct value to the organization; Intended
operational environment and observed usage patterns; Potential positive
and
negative impacts to individuals, public safety, groups, communities,
organizations, democratic institutions, and the physical environment;
Social
norms and expectations.
Harmful Bias and Homogenization
MP-1.1-003
Document risk measurement plans to address identified risks. Plans may
include, as applicable: Individual and group cognitive biases (e.g.,
confirmation
bias, funding bias, groupthink) for AI Actors involved in the design,
implementation, and use of GAI systems; Known past GAI system incidents
and
failure modes; In-context use and foreseeable misuse, abuse, and
off-label use;
Over reliance on quantitative metrics and methodologies without
sufficient
awareness of their limitations in the context(s) of use; Standard
measurement
and structured human feedback approaches; Anticipated human-AI
configurations.
Human-AI Configuration; Harmful
Bias and Homogenization;
Dangerous, Violent, or Hateful
Content
MP-1.1-004
Identify and document foreseeable illegal uses or applications of the
GAI system
that surpass organizational risk tolerances.
CBRN Information or Capabilities;
Dangerous, Violent, or Hateful
Content; Obscene, Degrading,
and/or Abusive Content
AI Actor Tasks: AI Deployment
MAP 1.2: Interdisciplinary AI Actors, competencies, skills, and
capacities for establishing context reflect demographic diversity and
broad domain and user experience expertise, and their participation is
documented. Opportunities for interdisciplinary
collaboration are prioritized.
Action ID
Suggested Action
GAI Risks
MP-1.2-001
Establish and empower interdisciplinary teams that reflect a wide range
of
capabilities, competencies, demographic groups, domain expertise,
educational
backgrounds, lived experiences, professions, and skills across the
enterprise to
inform and conduct risk measurement and management functions.
Human-AI Configuration; Harmful
Bias and Homogenization
MP-1.2-002
Verify that data or benchmarks used in risk measurement, and users,
participants, or subjects involved in structured GAI public feedback
exercises
are representative of diverse in-context user populations.
Human-AI Configuration; Harmful
Bias and Homogenization
AI Actor Tasks: AI Deployment
- >-
49
early lifecycle TEVV approaches are developed and matured for GAI,
organizations may use
recommended “pre-deployment testing” practices to measure performance,
capabilities, limits, risks,
and impacts. This section describes risk measurement and estimation as
part of pre-deployment TEVV,
and examines the state of play for pre-deployment testing
methodologies.
Limitations of Current Pre-deployment Test Approaches
Currently available pre-deployment TEVV processes used for GAI
applications may be inadequate, non-
systematically applied, or fail to reflect or mismatched to deployment
contexts. For example, the
anecdotal testing of GAI system capabilities through video games or
standardized tests designed for
humans (e.g., intelligence tests, professional licensing exams) does not
guarantee GAI system validity or
reliability in those domains. Similarly, jailbreaking or prompt
engineering tests may not systematically
assess validity or reliability risks.
Measurement gaps can arise from mismatches between laboratory and
real-world settings. Current
testing approaches often remain focused on laboratory conditions or
restricted to benchmark test
datasets and in silico techniques that may not extrapolate well to—or
directly assess GAI impacts in real-
world conditions. For example, current measurement gaps for GAI make it
difficult to precisely estimate
its potential ecosystem-level or longitudinal risks and related
political, social, and economic impacts.
Gaps between benchmarks and real-world use of GAI systems may likely be
exacerbated due to prompt
sensitivity and broad heterogeneity of contexts of use.
A.1.5. Structured Public Feedback
Structured public feedback can be used to evaluate whether GAI systems
are performing as intended
and to calibrate and verify traditional measurement methods. Examples of
structured feedback include,
but are not limited to:
•
Participatory Engagement Methods: Methods used to solicit feedback from
civil society groups,
affected communities, and users, including focus groups, small user
studies, and surveys.
•
Field Testing: Methods used to determine how people interact with,
consume, use, and make
sense of AI-generated information, and subsequent actions and effects,
including UX, usability,
and other structured, randomized experiments.
•
AI Red-teaming: A structured testing exercise used to probe an AI system
to find flaws and
vulnerabilities such as inaccurate, harmful, or discriminatory outputs,
often in a controlled
environment and in collaboration with system developers.
Information gathered from structured public feedback can inform design,
implementation, deployment
approval, maintenance, or decommissioning decisions. Results and
insights gleaned from these exercises
can serve multiple purposes, including improving data quality and
preprocessing, bolstering governance
decision making, and enhancing system documentation and debugging
practices. When implementing
feedback activities, organizations should follow human subjects research
requirements and best
practices such as informed consent and subject compensation.
- >-
ABOUT THIS FRAMEWORK
The Blueprint for an AI Bill of Rights is a set of five principles and
associated practices to help guide the
design, use, and deployment of automated systems to protect the rights
of the American public in the age of
artificial intel-ligence. Developed through extensive consultation with
the American public, these principles are
a blueprint for building and deploying automated systems that are
aligned with democratic values and protect
civil rights, civil liberties, and privacy. The Blueprint for an AI Bill
of Rights includes this Foreword, the five
principles, notes on Applying the The Blueprint for an AI Bill of
Rights, and a Technical Companion that gives
concrete steps that can be taken by many kinds of organizations—from
governments at all levels to companies of
all sizes—to uphold these values. Experts from across the private
sector, governments, and international
consortia have published principles and frameworks to guide the
responsible use of automated systems; this
framework provides a national values statement and toolkit that is
sector-agnostic to inform building these
protections into policy, practice, or the technological design process.
Where existing law or policy—such as
sector-specific privacy laws and oversight requirements—do not already
provide guidance, the Blueprint for an
AI Bill of Rights should be used to inform policy decisions.
LISTENING TO THE AMERICAN PUBLIC
The White House Office of Science and Technology Policy has led a
year-long process to seek and distill input
from people across the country—from impacted communities and industry
stakeholders to technology develop-
ers and other experts across fields and sectors, as well as policymakers
throughout the Federal government—on
the issue of algorithmic and data-driven harms and potential remedies.
Through panel discussions, public listen-
ing sessions, meetings, a formal request for information, and input to a
publicly accessible and widely-publicized
email address, people throughout the United States, public servants
across Federal agencies, and members of the
international community spoke up about both the promises and potential
harms of these technologies, and
played a central role in shaping the Blueprint for an AI Bill of Rights.
The core messages gleaned from these
discussions include that AI has transformative potential to improve
Americans’ lives, and that preventing the
harms of these technologies is both necessary and achievable. The
Appendix includes a full list of public engage-
ments.
4
- source_sentence: >-
What are the suggested actions for establishing transparency policies
related to GAI applications?
sentences:
- >-
42
MG-2.4-002
Establish and maintain procedures for escalating GAI system incidents to
the
organizational risk management authority when specific criteria for
deactivation
or disengagement is met for a particular context of use or for the GAI
system as a
whole.
Information Security
MG-2.4-003
Establish and maintain procedures for the remediation of issues which
trigger
incident response processes for the use of a GAI system, and provide
stakeholders
timelines associated with the remediation plan.
Information Security
MG-2.4-004 Establish and regularly review specific criteria that warrants
the deactivation of
GAI systems in accordance with set risk tolerances and appetites.
Information Security
AI Actor Tasks: AI Deployment, Governance and Oversight, Operation and
Monitoring
MANAGE 3.1: AI risks and benefits from third-party resources are
regularly monitored, and risk controls are applied and
documented.
Action ID
Suggested Action
GAI Risks
MG-3.1-001
Apply organizational risk tolerances and controls (e.g., acquisition
and
procurement processes; assessing personnel credentials and
qualifications,
performing background checks; filtering GAI input and outputs, grounding,
fine
tuning, retrieval-augmented generation) to third-party GAI resources:
Apply
organizational risk tolerance to the utilization of third-party datasets
and other
GAI resources; Apply organizational risk tolerances to fine-tuned
third-party
models; Apply organizational risk tolerance to existing third-party
models
adapted to a new domain; Reassess risk measurements after fine-tuning
third-
party GAI models.
Value Chain and Component
Integration; Intellectual Property
MG-3.1-002
Test GAI system value chain risks (e.g., data poisoning, malware, other
software
and hardware vulnerabilities; labor practices; data privacy and
localization
compliance; geopolitical alignment).
Data Privacy; Information Security;
Value Chain and Component
Integration; Harmful Bias and
Homogenization
MG-3.1-003
Re-assess model risks after fine-tuning or retrieval-augmented
generation
implementation and for any third-party GAI models deployed for
applications
and/or use cases that were not evaluated in initial testing.
Value Chain and Component
Integration
MG-3.1-004
Take reasonable measures to review training data for CBRN information,
and
intellectual property, and where appropriate, remove it. Implement
reasonable
measures to prevent, flag, or take other action in response to outputs
that
reproduce particular training data (e.g., plagiarized, trademarked,
patented,
licensed content or trade secret material).
Intellectual Property; CBRN
Information or Capabilities
- >-
DATA PRIVACY
EXTRA PROTECTIONS FOR DATA RELATED TO SENSITIVE
DOMAINS
•
Continuous positive airway pressure machines gather data for medical
purposes, such as diagnosing sleep
apnea, and send usage data to a patient’s insurance company, which may
subsequently deny coverage for the
device based on usage data. Patients were not aware that the data would
be used in this way or monitored
by anyone other than their doctor.70
•
A department store company used predictive analytics applied to
collected consumer data to determine that a
teenage girl was pregnant, and sent maternity clothing ads and other
baby-related advertisements to her
house, revealing to her father that she was pregnant.71
•
School audio surveillance systems monitor student conversations to
detect potential "stress indicators" as
a warning of potential violence.72 Online proctoring systems claim to
detect if a student is cheating on an
exam using biometric markers.73 These systems have the potential to
limit student freedom to express a range
of emotions at school and may inappropriately flag students with
disabilities who need accommodations or
use screen readers or dictation software as cheating.74
•
Location data, acquired from a data broker, can be used to identify
people who visit abortion clinics.75
•
Companies collect student data such as demographic information, free or
reduced lunch status, whether
they've used drugs, or whether they've expressed interest in LGBTQI+
groups, and then use that data to
forecast student success.76 Parents and education experts have expressed
concern about collection of such
sensitive data without express parental consent, the lack of
transparency in how such data is being used, and
the potential for resulting discriminatory impacts.
• Many employers transfer employee data to third party job verification
services. This information is then used
by potential future employers, banks, or landlords. In one case, a
former employee alleged that a
company supplied false data about her job title which resulted in a job
offer being revoked.77
37
- >-
14
GOVERN 1.2: The characteristics of trustworthy AI are integrated into
organizational policies, processes, procedures, and practices.
Action ID
Suggested Action
GAI Risks
GV-1.2-001
Establish transparency policies and processes for documenting the origin
and
history of training data and generated data for GAI applications to
advance digital
content transparency, while balancing the proprietary nature of
training
approaches.
Data Privacy; Information
Integrity; Intellectual Property
GV-1.2-002
Establish policies to evaluate risk-relevant capabilities of GAI and
robustness of
safety measures, both prior to deployment and on an ongoing basis,
through
internal and external evaluations.
CBRN Information or Capabilities;
Information Security
AI Actor Tasks: Governance and Oversight
GOVERN 1.3: Processes, procedures, and practices are in place to
determine the needed level of risk management activities based
on the organization’s risk tolerance.
Action ID
Suggested Action
GAI Risks
GV-1.3-001
Consider the following factors when updating or defining risk tiers for
GAI: Abuses
and impacts to information integrity; Dependencies between GAI and other
IT or
data systems; Harm to fundamental rights or public safety; Presentation
of
obscene, objectionable, offensive, discriminatory, invalid or untruthful
output;
Psychological impacts to humans (e.g., anthropomorphization,
algorithmic
aversion, emotional entanglement); Possibility for malicious use;
Whether the
system introduces significant new security vulnerabilities; Anticipated
system
impact on some groups compared to others; Unreliable decision making
capabilities, validity, adaptability, and variability of GAI system
performance over
time.
Information Integrity; Obscene,
Degrading, and/or Abusive
Content; Value Chain and
Component Integration; Harmful
Bias and Homogenization;
Dangerous, Violent, or Hateful
Content; CBRN Information or
Capabilities
GV-1.3-002
Establish minimum thresholds for performance or assurance criteria and
review as
part of deployment approval (“go/”no-go”) policies, procedures, and
processes,
with reviewed processes and approval thresholds reflecting measurement of
GAI
capabilities and risks.
CBRN Information or Capabilities;
Confabulation; Dangerous,
Violent, or Hateful Content
GV-1.3-003
Establish a test plan and response policy, before developing highly
capable models,
to periodically evaluate whether the model may misuse CBRN information
or
capabilities and/or offensive cyber capabilities.
CBRN Information or Capabilities;
Information Security
- source_sentence: >-
What methods are suggested for recording and integrating structured
feedback about content provenance from various stakeholders in the context
of GAI systems?
sentences:
- >-
39
MS-3.3-004
Provide input for training materials about the capabilities and
limitations of GAI
systems related to digital content transparency for AI Actors, other
professionals, and the public about the societal impacts of AI and the
role of
diverse and inclusive content generation.
Human-AI Configuration;
Information Integrity; Harmful Bias
and Homogenization
MS-3.3-005
Record and integrate structured feedback about content provenance from
operators, users, and potentially impacted communities through the use
of
methods such as user research studies, focus groups, or community
forums.
Actively seek feedback on generated content quality and potential
biases.
Assess the general awareness among end users and impacted communities
about the availability of these feedback channels.
Human-AI Configuration;
Information Integrity; Harmful Bias
and Homogenization
AI Actor Tasks: AI Deployment, Affected Individuals and Communities,
End-Users, Operation and Monitoring, TEVV
MEASURE 4.2: Measurement results regarding AI system trustworthiness in
deployment context(s) and across the AI lifecycle are
informed by input from domain experts and relevant AI Actors to validate
whether the system is performing consistently as
intended. Results are documented.
Action ID
Suggested Action
GAI Risks
MS-4.2-001
Conduct adversarial testing at a regular cadence to map and measure GAI
risks,
including tests to address attempts to deceive or manipulate the
application of
provenance techniques or other misuses. Identify vulnerabilities and
understand potential misuse scenarios and unintended outputs.
Information Integrity; Information
Security
MS-4.2-002
Evaluate GAI system performance in real-world scenarios to observe its
behavior in practical environments and reveal issues that might not
surface in
controlled and optimized testing environments.
Human-AI Configuration;
Confabulation; Information
Security
MS-4.2-003
Implement interpretability and explainability methods to evaluate GAI
system
decisions and verify alignment with intended purpose.
Information Integrity; Harmful Bias
and Homogenization
MS-4.2-004
Monitor and document instances where human operators or other systems
override the GAI's decisions. Evaluate these cases to understand if the
overrides
are linked to issues related to content provenance.
Information Integrity
MS-4.2-005
Verify and document the incorporation of results of structured public
feedback
exercises into design, implementation, deployment approval
(“go”/“no-go”
decisions), monitoring, and decommission decisions.
Human-AI Configuration;
Information Security
AI Actor Tasks: AI Deployment, Domain Experts, End-Users, Operation and
Monitoring, TEVV
- >-
46
MG-4.3-003
Report GAI incidents in compliance with legal and regulatory
requirements (e.g.,
HIPAA breach reporting, e.g., OCR (2023) or NHTSA (2022) autonomous
vehicle
crash reporting requirements.
Information Security; Data Privacy
AI Actor Tasks: AI Deployment, Affected Individuals and Communities,
Domain Experts, End-Users, Human Factors, Operation and
Monitoring
- >-
ENDNOTES
107. Centers for Medicare & Medicaid Services. Biden-Harris
Administration Quadruples the Number
of Health Care Navigators Ahead of HealthCare.gov Open Enrollment
Period. Aug. 27, 2021.
https://www.cms.gov/newsroom/press-releases/biden-harris-administration-quadruples-number
health-care-navigators-ahead-healthcaregov-open
108. See, e.g., McKinsey & Company. The State of Customer Care in 2022.
July 8, 2022. https://
www.mckinsey.com/business-functions/operations/our-insights/the-state-of-customer-care-in-2022;
Sara Angeles. Customer Service Solutions for Small Businesses. Business
News Daily.
Jun. 29, 2022.
https://www.businessnewsdaily.com/7575-customer-service-solutions.html
109. Mike Hughes. Are We Getting The Best Out Of Our Bots?
Co-Intelligence Between Robots &
Humans. Forbes. Jul. 14, 2022.
https://www.forbes.com/sites/mikehughes1/2022/07/14/are-we-getting-the-best-out-of-our-bots-co
intelligence-between-robots--humans/?sh=16a2bd207395
110. Rachel Orey and Owen Bacskai. The Low Down on Ballot Curing. Nov.
04, 2020. https://
bipartisanpolicy.org/blog/the-low-down-on-ballot-curing/; Zahavah Levine
and Thea Raymond-
Seidel. Mail Voting Litigation in 2020, Part IV: Verifying Mail Ballots.
Oct. 29, 2020.
https://www.lawfareblog.com/mail-voting-litigation-2020-part-iv-verifying-mail-ballots
111. National Conference of State Legislatures. Table 15: States With
Signature Cure Processes. Jan. 18,
2022.
https://www.ncsl.org/research/elections-and-campaigns/vopp-table-15-states-that-permit-voters-to
correct-signature-discrepancies.aspx
112. White House Office of Science and Technology Policy. Join the
Effort to Create A Bill of Rights for
an Automated Society. Nov. 10, 2021.
https://www.whitehouse.gov/ostp/news-updates/2021/11/10/join-the-effort-to-create-a-bill-of
rights-for-an-automated-society/
113. White House Office of Science and Technology Policy. Notice of
Request for Information (RFI) on
Public and Private Sector Uses of Biometric Technologies. Issued Oct. 8,
2021.
https://www.federalregister.gov/documents/2021/10/08/2021-21975/notice-of-request-for
information-rfi-on-public-and-private-sector-uses-of-biometric-technologies
114. National Artificial Intelligence Initiative Office. Public Input on
Public and Private Sector Uses of
Biometric Technologies. Accessed Apr. 19, 2022.
https://www.ai.gov/86-fr-56300-responses/
115. Thomas D. Olszewski, Lisa M. Van Pay, Javier F. Ortiz, Sarah E.
Swiersz, and Laurie A. Dacus.
Synopsis of Responses to OSTP’s Request for Information on the Use and
Governance of Biometric
Technologies in the Public and Private Sectors. Science and Technology
Policy Institute. Mar. 2022.
https://www.ida.org/-/media/feature/publications/s/sy/synopsis-of-responses-to-request-for
information-on-the-use-and-governance-of-biometric-technologies/ida-document-d-33070.ashx
73
SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2
This is a sentence-transformers model finetuned from sentence-transformers/all-MiniLM-L6-v2. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
Model Details
Model Description
- Model Type: Sentence Transformer
- Base model: sentence-transformers/all-MiniLM-L6-v2
- Maximum Sequence Length: 256 tokens
- Output Dimensionality: 384 tokens
- Similarity Function: Cosine Similarity
Model Sources
- Documentation: Sentence Transformers Documentation
- Repository: Sentence Transformers on GitHub
- Hugging Face: Sentence Transformers on Hugging Face
Full Model Architecture
SentenceTransformer(
(0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel
(1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
(2): Normalize()
)
Usage
Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
pip install -U sentence-transformers
Then you can load this model and run inference.
from sentence_transformers import SentenceTransformer
# Download from the 🤗 Hub
model = SentenceTransformer("danicafisher/dfisher-sentence-transformer-fine-tuned")
# Run inference
sentences = [
'What methods are suggested for recording and integrating structured feedback about content provenance from various stakeholders in the context of GAI systems?',
"39 \nMS-3.3-004 \nProvide input for training materials about the capabilities and limitations of GAI \nsystems related to digital content transparency for AI Actors, other \nprofessionals, and the public about the societal impacts of AI and the role of \ndiverse and inclusive content generation. \nHuman-AI Configuration; \nInformation Integrity; Harmful Bias \nand Homogenization \nMS-3.3-005 \nRecord and integrate structured feedback about content provenance from \noperators, users, and potentially impacted communities through the use of \nmethods such as user research studies, focus groups, or community forums. \nActively seek feedback on generated content quality and potential biases. \nAssess the general awareness among end users and impacted communities \nabout the availability of these feedback channels. \nHuman-AI Configuration; \nInformation Integrity; Harmful Bias \nand Homogenization \nAI Actor Tasks: AI Deployment, Affected Individuals and Communities, End-Users, Operation and Monitoring, TEVV \n \nMEASURE 4.2: Measurement results regarding AI system trustworthiness in deployment context(s) and across the AI lifecycle are \ninformed by input from domain experts and relevant AI Actors to validate whether the system is performing consistently as \nintended. Results are documented. \nAction ID \nSuggested Action \nGAI Risks \nMS-4.2-001 \nConduct adversarial testing at a regular cadence to map and measure GAI risks, \nincluding tests to address attempts to deceive or manipulate the application of \nprovenance techniques or other misuses. Identify vulnerabilities and \nunderstand potential misuse scenarios and unintended outputs. \nInformation Integrity; Information \nSecurity \nMS-4.2-002 \nEvaluate GAI system performance in real-world scenarios to observe its \nbehavior in practical environments and reveal issues that might not surface in \ncontrolled and optimized testing environments. \nHuman-AI Configuration; \nConfabulation; Information \nSecurity \nMS-4.2-003 \nImplement interpretability and explainability methods to evaluate GAI system \ndecisions and verify alignment with intended purpose. \nInformation Integrity; Harmful Bias \nand Homogenization \nMS-4.2-004 \nMonitor and document instances where human operators or other systems \noverride the GAI's decisions. Evaluate these cases to understand if the overrides \nare linked to issues related to content provenance. \nInformation Integrity \nMS-4.2-005 \nVerify and document the incorporation of results of structured public feedback \nexercises into design, implementation, deployment approval (“go”/“no-go” \ndecisions), monitoring, and decommission decisions. \nHuman-AI Configuration; \nInformation Security \nAI Actor Tasks: AI Deployment, Domain Experts, End-Users, Operation and Monitoring, TEVV",
'46 \nMG-4.3-003 \nReport GAI incidents in compliance with legal and regulatory requirements (e.g., \nHIPAA breach reporting, e.g., OCR (2023) or NHTSA (2022) autonomous vehicle \ncrash reporting requirements. \nInformation Security; Data Privacy \nAI Actor Tasks: AI Deployment, Affected Individuals and Communities, Domain Experts, End-Users, Human Factors, Operation and \nMonitoring',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]
# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
Training Details
Training Dataset
Unnamed Dataset
- Size: 274 training samples
- Columns:
sentence_0andsentence_1 - Approximate statistics based on the first 274 samples:
sentence_0 sentence_1 type string string details - min: 12 tokens
- mean: 22.67 tokens
- max: 38 tokens
- min: 21 tokens
- mean: 245.27 tokens
- max: 256 tokens
- Samples:
sentence_0 sentence_1 How does the Executive Order on Advancing Racial Equity define 'equity' and 'underserved communities'?ENDNOTES
47. Darshali A. Vyas et al., Hidden in Plain Sight – Reconsidering the Use of Race Correction in Clinical
Algorithms, 383 N. Engl. J. Med.874, 876-78 (Aug. 27, 2020), https://www.nejm.org/doi/full/10.1056/
NEJMms2004740.
48. The definitions of 'equity' and 'underserved communities' can be found in the Definitions section of
this framework as well as in Section 2 of The Executive Order On Advancing Racial Equity and Support
for Underserved Communities Through the Federal Government. https://www.whitehouse.gov/
briefing-room/presidential-actions/2021/01/20/executive-order-advancing-racial-equity-and-support
for-underserved-communities-through-the-federal-government/
49. Id.
50. Various organizations have offered proposals for how such assessments might be designed. See, e.g.,
Emanuel Moss, Elizabeth Anne Watkins, Ranjit Singh, Madeleine Clare Elish, and Jacob Metcalf.
Assembling Accountability: Algorithmic Impact Assessment for the Public Interest. Data & Society
Research Institute Report. June 29, 2021. https://datasociety.net/library/assembling-accountability
algorithmic-impact-assessment-for-the-public-interest/; Nicol Turner Lee, Paul Resnick, and Genie
Barton. Algorithmic bias detection and mitigation: Best practices and policies to reduce consumer harms.
Brookings Report. May 22, 2019.
https://www.brookings.edu/research/algorithmic-bias-detection-and-mitigation-best-practices-and
policies-to-reduce-consumer-harms/; Andrew D. Selbst. An Institutional View Of Algorithmic Impact
Assessments. Harvard Journal of Law & Technology. June 15, 2021. https://ssrn.com/abstract=3867634;
Dillon Reisman, Jason Schultz, Kate Crawford, and Meredith Whittaker. Algorithmic Impact
Assessments: A Practical Framework for Public Agency Accountability. AI Now Institute Report. April
2018. https://ainowinstitute.org/aiareport2018.pdf
51. Department of Justice. Justice Department Announces New Initiative to Combat Redlining. Oct. 22,
2021. https://www.justice.gov/opa/pr/justice-department-announces-new-initiative-combat-redlining
52. PAVE Interagency Task Force on Property Appraisal and Valuation Equity. Action Plan to Advance
Property Appraisal and Valuation Equity: Closing the Racial Wealth Gap by Addressing Mis-valuations for
Families and Communities of Color. March 2022. https://pave.hud.gov/sites/pave.hud.gov/files/
documents/PAVEActionPlan.pdf
53. U.S. Equal Employment Opportunity Commission. The Americans with Disabilities Act and the Use of
Software, Algorithms, and Artificial Intelligence to Assess Job Applicants and Employees. EEOC
NVTA-2022-2. May 12, 2022. https://www.eeoc.gov/laws/guidance/americans-disabilities-act-and-use
software-algorithms-and-artificial-intelligence; U.S. Department of Justice. Algorithms, Artificial
Intelligence, and Disability Discrimination in Hiring. May 12, 2022. https://beta.ada.gov/resources/ai
guidance/
54. Ziad Obermeyer, Brian Powers, Christine Vogeli, and Sendhil Mullainathan. Dissecting racial bias in
an algorithm used to manage the health of populations. Science. Vol. 366, No. 6464. Oct. 25, 2019. https://
www.science.org/doi/10.1126/science.aax2342
55. Data & Trust Alliance. Algorithmic Bias Safeguards for Workforce: Overview. Jan. 2022. https://
dataandtrustalliance.org/Algorithmic_Bias_Safeguards_for_Workforce_Overview.pdf
56. Section 508.gov. IT Accessibility Laws and Policies. Access Board. https://www.section508.gov/
manage/laws-and-policies/
67What are the key expectations for automated systems as outlined in the context?HUMAN ALTERNATIVES,
CONSIDERATION, AND
FALLBACK
WHAT SHOULD BE EXPECTED OF AUTOMATED SYSTEMS
The expectations for automated systems are meant to serve as a blueprint for the development of additional
technical standards and practices that are tailored for particular sectors and contexts.
Equitable. Consideration should be given to ensuring outcomes of the fallback and escalation system are
equitable when compared to those of the automated system and such that the fallback and escalation
system provides equitable access to underserved communities.105
Timely. Human consideration and fallback are only useful if they are conducted and concluded in a
timely manner. The determination of what is timely should be made relative to the specific automated
system, and the review system should be staffed and regularly assessed to ensure it is providing timely
consideration and fallback. In time-critical systems, this mechanism should be immediately available or,
where possible, available before the harm occurs. Time-critical systems include, but are not limited to,
voting-related systems, automated building access and other access systems, systems that form a critical
component of healthcare, and systems that have the ability to withhold wages or otherwise cause
immediate financial penalties.
Effective. The organizational structure surrounding processes for consideration and fallback should
be designed so that if the human decision-maker charged with reassessing a decision determines that it
should be overruled, the new decision will be effectively enacted. This includes ensuring that the new
decision is entered into the automated system throughout its components, any previous repercussions from
the old decision are also overturned, and safeguards are put in place to help ensure that future decisions do
not result in the same errors.
Maintained. The human consideration and fallback process and any associated automated processes
should be maintained and supported as long as the relevant automated system continues to be in use.
Institute training, assessment, and oversight to combat automation bias and ensure any
human-based components of a system are effective.
Training and assessment. Anyone administering, interacting with, or interpreting the outputs of an auto
mated system should receive training in that system, including how to properly interpret outputs of a system
in light of its intended purpose and in how to mitigate the effects of automation bias. The training should reoc
cur regularly to ensure it is up to date with the system and to ensure the system is used appropriately. Assess
ment should be ongoing to ensure that the use of the system with human involvement provides for appropri
ate results, i.e., that the involvement of people does not invalidate the system's assessment as safe and effective
or lead to algorithmic discrimination.
Oversight. Human-based systems have the potential for bias, including automation bias, as well as other
concerns that may limit their effectiveness. The results of assessments of the efficacy and potential bias of
such human-based systems should be overseen by governance structures that have the potential to update the
operation of the human-based system in order to mitigate these effects.
50What is the focus of the report titled "Assembling Accountability: Algorithmic Impact Assessment for the Public Interest" by Emanuel Moss and others?ENDNOTES
47. Darshali A. Vyas et al., Hidden in Plain Sight – Reconsidering the Use of Race Correction in Clinical
Algorithms, 383 N. Engl. J. Med.874, 876-78 (Aug. 27, 2020), https://www.nejm.org/doi/full/10.1056/
NEJMms2004740.
48. The definitions of 'equity' and 'underserved communities' can be found in the Definitions section of
this framework as well as in Section 2 of The Executive Order On Advancing Racial Equity and Support
for Underserved Communities Through the Federal Government. https://www.whitehouse.gov/
briefing-room/presidential-actions/2021/01/20/executive-order-advancing-racial-equity-and-support
for-underserved-communities-through-the-federal-government/
49. Id.
50. Various organizations have offered proposals for how such assessments might be designed. See, e.g.,
Emanuel Moss, Elizabeth Anne Watkins, Ranjit Singh, Madeleine Clare Elish, and Jacob Metcalf.
Assembling Accountability: Algorithmic Impact Assessment for the Public Interest. Data & Society
Research Institute Report. June 29, 2021. https://datasociety.net/library/assembling-accountability
algorithmic-impact-assessment-for-the-public-interest/; Nicol Turner Lee, Paul Resnick, and Genie
Barton. Algorithmic bias detection and mitigation: Best practices and policies to reduce consumer harms.
Brookings Report. May 22, 2019.
https://www.brookings.edu/research/algorithmic-bias-detection-and-mitigation-best-practices-and
policies-to-reduce-consumer-harms/; Andrew D. Selbst. An Institutional View Of Algorithmic Impact
Assessments. Harvard Journal of Law & Technology. June 15, 2021. https://ssrn.com/abstract=3867634;
Dillon Reisman, Jason Schultz, Kate Crawford, and Meredith Whittaker. Algorithmic Impact
Assessments: A Practical Framework for Public Agency Accountability. AI Now Institute Report. April
2018. https://ainowinstitute.org/aiareport2018.pdf
51. Department of Justice. Justice Department Announces New Initiative to Combat Redlining. Oct. 22,
2021. https://www.justice.gov/opa/pr/justice-department-announces-new-initiative-combat-redlining
52. PAVE Interagency Task Force on Property Appraisal and Valuation Equity. Action Plan to Advance
Property Appraisal and Valuation Equity: Closing the Racial Wealth Gap by Addressing Mis-valuations for
Families and Communities of Color. March 2022. https://pave.hud.gov/sites/pave.hud.gov/files/
documents/PAVEActionPlan.pdf
53. U.S. Equal Employment Opportunity Commission. The Americans with Disabilities Act and the Use of
Software, Algorithms, and Artificial Intelligence to Assess Job Applicants and Employees. EEOC
NVTA-2022-2. May 12, 2022. https://www.eeoc.gov/laws/guidance/americans-disabilities-act-and-use
software-algorithms-and-artificial-intelligence; U.S. Department of Justice. Algorithms, Artificial
Intelligence, and Disability Discrimination in Hiring. May 12, 2022. https://beta.ada.gov/resources/ai
guidance/
54. Ziad Obermeyer, Brian Powers, Christine Vogeli, and Sendhil Mullainathan. Dissecting racial bias in
an algorithm used to manage the health of populations. Science. Vol. 366, No. 6464. Oct. 25, 2019. https://
www.science.org/doi/10.1126/science.aax2342
55. Data & Trust Alliance. Algorithmic Bias Safeguards for Workforce: Overview. Jan. 2022. https://
dataandtrustalliance.org/Algorithmic_Bias_Safeguards_for_Workforce_Overview.pdf
56. Section 508.gov. IT Accessibility Laws and Policies. Access Board. https://www.section508.gov/
manage/laws-and-policies/
67 - Loss:
MultipleNegativesRankingLosswith these parameters:{ "scale": 20.0, "similarity_fct": "cos_sim" }
Training Hyperparameters
Non-Default Hyperparameters
per_device_train_batch_size: 16per_device_eval_batch_size: 16multi_dataset_batch_sampler: round_robin
All Hyperparameters
Click to expand
overwrite_output_dir: Falsedo_predict: Falseeval_strategy: noprediction_loss_only: Trueper_device_train_batch_size: 16per_device_eval_batch_size: 16per_gpu_train_batch_size: Noneper_gpu_eval_batch_size: Nonegradient_accumulation_steps: 1eval_accumulation_steps: Nonetorch_empty_cache_steps: Nonelearning_rate: 5e-05weight_decay: 0.0adam_beta1: 0.9adam_beta2: 0.999adam_epsilon: 1e-08max_grad_norm: 1num_train_epochs: 3max_steps: -1lr_scheduler_type: linearlr_scheduler_kwargs: {}warmup_ratio: 0.0warmup_steps: 0log_level: passivelog_level_replica: warninglog_on_each_node: Truelogging_nan_inf_filter: Truesave_safetensors: Truesave_on_each_node: Falsesave_only_model: Falserestore_callback_states_from_checkpoint: Falseno_cuda: Falseuse_cpu: Falseuse_mps_device: Falseseed: 42data_seed: Nonejit_mode_eval: Falseuse_ipex: Falsebf16: Falsefp16: Falsefp16_opt_level: O1half_precision_backend: autobf16_full_eval: Falsefp16_full_eval: Falsetf32: Nonelocal_rank: 0ddp_backend: Nonetpu_num_cores: Nonetpu_metrics_debug: Falsedebug: []dataloader_drop_last: Falsedataloader_num_workers: 0dataloader_prefetch_factor: Nonepast_index: -1disable_tqdm: Falseremove_unused_columns: Truelabel_names: Noneload_best_model_at_end: Falseignore_data_skip: Falsefsdp: []fsdp_min_num_params: 0fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}fsdp_transformer_layer_cls_to_wrap: Noneaccelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}deepspeed: Nonelabel_smoothing_factor: 0.0optim: adamw_torchoptim_args: Noneadafactor: Falsegroup_by_length: Falselength_column_name: lengthddp_find_unused_parameters: Noneddp_bucket_cap_mb: Noneddp_broadcast_buffers: Falsedataloader_pin_memory: Truedataloader_persistent_workers: Falseskip_memory_metrics: Trueuse_legacy_prediction_loop: Falsepush_to_hub: Falseresume_from_checkpoint: Nonehub_model_id: Nonehub_strategy: every_savehub_private_repo: Falsehub_always_push: Falsegradient_checkpointing: Falsegradient_checkpointing_kwargs: Noneinclude_inputs_for_metrics: Falseeval_do_concat_batches: Truefp16_backend: autopush_to_hub_model_id: Nonepush_to_hub_organization: Nonemp_parameters:auto_find_batch_size: Falsefull_determinism: Falsetorchdynamo: Noneray_scope: lastddp_timeout: 1800torch_compile: Falsetorch_compile_backend: Nonetorch_compile_mode: Nonedispatch_batches: Nonesplit_batches: Noneinclude_tokens_per_second: Falseinclude_num_input_tokens_seen: Falseneftune_noise_alpha: Noneoptim_target_modules: Nonebatch_eval_metrics: Falseeval_on_start: Falseeval_use_gather_object: Falsebatch_sampler: batch_samplermulti_dataset_batch_sampler: round_robin
Framework Versions
- Python: 3.11.9
- Sentence Transformers: 3.1.1
- Transformers: 4.44.2
- PyTorch: 2.4.1
- Accelerate: 0.34.2
- Datasets: 3.0.0
- Tokenizers: 0.19.1
Citation
BibTeX
Sentence Transformers
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
MultipleNegativesRankingLoss
@misc{henderson2017efficient,
title={Efficient Natural Language Response Suggestion for Smart Reply},
author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
year={2017},
eprint={1705.00652},
archivePrefix={arXiv},
primaryClass={cs.CL}
}