HOMEWORK.

Assignment 1: due Wednesday, April 3, 2013, in class.
Please hand in one page answering the following questions.
A. In one paragraph, describe your background and interest in taking this course. Indicate any specific topics you are interested in.
B. In one paragraph,
(a) if you are from a department other than Mathematics, describe a problem where you would like to be able to use a stochastic process model; then describe in words what X(t), the state of the process at time t would represent. Specify the time index set and the state space for your process. Try to formulate questions that you would like to be able to answer for this process. Try to draw a typical sample path.
(b) for Mathematics students, or those who do not have a problem in mind, describe a phenomenon in the real world where you think a stochastic process model would be useful and then answer the same questions as in (a) above.
C. Describe the state space for a stochastic process that describes the random evolution in time of a finite DNA segment, where mutations, insertions and deletions of sites may occur. (You may assume that there are 4 possible values at each site, A, C, G, T.)
For solutions to part C, click here. (You will need the username and password for the class web access.)

Please submit all homework assignments after the first one in the homework box located in the basement of AP&M. The box will have the TA's name on it: David Lipshutz and the class: Math 285. No late homework will be accepted.

Assignment 2: due Friday, April 12, 2013, at 3pm.
Problems 1.1.2, last half of 1.1.3 (the part that starts on page 9), 1.1.4, 1.1.6, 1.2.1, 1.3.2, 1.3.3, 1.3.4 from the text by Norris.
Hint for problem 1.3.4: you may use the fact (due to Euler) that the sum from n=1 to infinity of 1/ n^2 is pi^2/6.
For solutions to Homework 2, click here. (You will need the username and password for the class to access these solutions.)

Assignment 3: due Friday, April 26, 2013 at 3pm.
Problems 1.5.1, 1.5.4 (note that the third part of this problem is quite challenging), 1.7.1 (here invariant distribution means stationary distribution), 1.8.2(b)(for this problem you will need to also solve 1.1.7b), 1.8.3. In addition, for the Markov chain in 1.8.2(b), find the long run probability that the Markov chain is in each of the states 1,2,3.
Also solve the following: Show that the Markov chain in Exercise 1.3.4 is transient and deduce that P(X(n) tends to infinity as n tends to infinity)=1 given X(0)=i for any i>=1.
Exercise 1.9.1 (b), (e)(assume S is finite), 1.10.1.
Consider the Markov chain X with transition matrix given by P in EXAMPLE 1.2.2 on Page 11 of Norris. Prove that T=inf{n>10: X(n) = 6} is a stopping time for the Markov chain when started from the state labelled 1.
For solutions to Homework 3, click here. (You will need the username and password for the class

Assignment 4: due Friday, May 3, 2013 at 3pm. Consider a Hidden Markov Model representation of a coin tossing experiment. Assume a two-state model (corresponding to two coins) with probability of output (of H or T) given by: P(H|Coin 1) = 0.5, P(T|Coin 1) = 0.5 and P(H|Coin 2)= 0.2, P(T|Coin 2)=0.8. Assume that the transition probability matrix for the Markov chain governing the coin state is: P(11) = 0.7, P(12)=0.3, P(21)=0.4, P(22) =0.6, where P(ij) denotes the probability that given that the coin is currently in state i, it is in state j at the next time step. Assume that initially the coin is equally likely to be coin 1 or 2.
(a) Suppose that you observe the sequence HHTTHTH. What is a/the most likely coin state sequence to generate this output?
(b) What is the probability that, given the observation sequence, it was generated entirely by coin 1?
For solutions to Homework 4, click here. (You will need the username and password for the class

Assignment 5: due Friday May 24, 2013 at 3pm.
Click here to see the pdf file with the homework assignment.

Assignment 6: due Friday June 7, 2013 at 3pm (this is the last assignment).
Click here to see the pdf file with the homework assignment 6. Note: for problem 1, you can use S_n as well as S_n squared to make a martingale.