But to measure the true entropy of a language, we need to consider sequences of infinite length. If we think of a language as a stochastic process L that produces a sequence of words, and allow W to represent the sequence of words w 1 , . . . , w n , then L's entropy rate H(L) is defined as