To capture these three different roles, transformers introduce weight matrices W Q , W K , and W V . These weights will be used to project each input vector x i into a representation of its role as a key, query, or value.