an initial probability distribution over states. π i is the probability that the Markov chain will start in state i. Some states j may have π j = 0, meaning that they cannot be initial states. Also, n i=1 π i = 1 Before you go on, use the sample probabilities in Fig. 8.8a (with π = [0.1, 0.7, 0.2] ) to compute the probability of each of the following sequences: