Then to make effective use of these scores, we'll normalize them with a softmax to create a vector of weights, α i j , that indicates the proportional relevance of each input to the input element i that is the current focus of attention.