HOMEWORK.

Assignment 1: due Wednesday, April 4, 2007, in class.
Please hand in one page answering the following questions.
A. In one paragraph, describe your background and interest in taking this course. Indicate any specific topics you are interested in.
B. In one paragraph,
(a) if you are from a department other than Mathematics, describe a problem where you would like to be able to use a stochastic process model; then try to describe in words what X(t), the state of the process at time t would represent. Specify the time index set and the state space for your process. Try to formulate questions that you would like to be able to answer for this process. Try to draw a typical sample path.
(b) for Mathematics students, or those who do not have a problem in mind, describe a phenomenon in the real world where you think a stochastic process model would be useful and then answer the same questions as in (a) above.
C. Describe the state space for a stochastic process that describes the random evolution in time of a finite DNA segment, where mutations, insertions and deletions of sites may occur. (You may assume that there are 4 possible values at each site, A, C, G, T.)
Solutions to Assignment 1 part C.

Assignment 2: due Wednesday, April 11, 2007, in class.
Problems 1.1.2, last half of 1.1.3 (the part that starts on page 9), 1.1.6 from the text by Norris.
Solutions to Assignment 2.

Assignment 3: due Wednesday, April 18, 2007, in class.
Problems 1.2.1, 1.3.2, 1.3.4 from the text by Norris.
Hint for problem 1.3.4: you may use the fact (due to Euler) that the sum from n=1 to infinity of 1/n^2 is pi^2/6.
Solutions to Assignment 3.

Assignment 4: due Wednesday, May 2, 2007, in class.
Problems 1.5.1, 1.8.2(a)(for this problem you will need to also solve 1.1.7a). In addition, for the Markov chain in 1.8.2(a), find the long run probability that the Markov chain is in each of the states 1,2,3.
Also solve the following: Show that the Markov chain in Exercise 1.3.4 is transient and deduce that P(X(n) tends to infinity as n tends to infinity)=1 given X(0)=i for any i>=1.
Solutions for Assignment 4 are here.

Assignment 5: due Wednesday, May 16, 2007, in class.
1. Exercise 1.9.1 (b), (e)(assume S is finite).
2. Exercise 1.10.1.
3. Consider the Markov chain X with transition matrix given by P in EXAMPLE 1.2.2 on Page 11 of Norris. Prove that T=inf{n>10: X(n) = 6} is a stopping time for the Markov chain when started from the state labelled 1.
4. Consider a Hidden Markov Model representation of a coin tossing experiment. Assume a two-state model (corresponding to two coins) with probability of output (of H or T) given by: P(H|Coin 1) = 0.5, P(T|Coin 1) = 0.5 and P(H|Coin 2)= 0.25, P(T|Coin 2)=0.75. Assume that the transition probability matrix for the Markov chain governing the coin state is: P(11) = 0.7, P(12)=0.3, P(21)=0.4, P(22) =0.6, where P(ij) denotes the probability that given that the coin is currently in state i, it is in state j at the next time step. Assume that initially the coin is equally likely to be coin 1 or 2.
(a) Suppose that you observe the sequence HHTTTH. What is a/the most likely state sequence to generate this output?
(b) What is the probability that given the observation sequence it was generated entirely by coin 2?
Solutions to assignment 5 are here.

Assignment 6: due Monday, June 4, 2007, in class.
Click here for the homework assignment.
Solutions to Homework 6.

Assignment 7 (optional): due Tuesday, June 12, 2007 at 5pm.
Click here for the homework assignment.
Solutions to Assignment 7.