Similarly, a simple average of the vectors in a span is unlikely to be an optimal representation of a span since it treats all of a span's embeddings as equally important. For many applications, a more useful representation would be centered around the head of the phrase corresponding to the span. One method for getting at such information in the absence of a syntactic parse is to use a standard self-attention layer to generate a span representation.