MATH 285: STOCHASTIC PROCESSES (SPRING 2005)

MATH 285: STOCHASTIC PROCESSES (SPRING 2005)
Copyright c2005, R. J. Williams
DO NOT REPRODUCE WITHOUT PERMISSION

Please send notice of typos to williams@math.ucsd.edu

The start of a new main topic is indicated by highlighting the topic in red.

Lecture 1. Introduction to Stochastic Processes

Lecture 2. Ways of viewing stochastic processes. Discrete Time Markov Chains: Definition and Examples

Lecture 3. Discrete Time Markov Chains (cont.): Hitting Times

Lecture 4. Discrete Time Markov Chains (cont.): Hitting times and first step analysis

Lecture 5. Discrete Time Markov Chains (cont.): Long run behavior -- stationary and steady state distributions, classification of states, period

Lecture 6. Discrete Time Markov Chains (cont.): Long run behavior -- recurrence, transience, basic limit theorem

Lecture 7. Reversible Markov Chains, Markov Chain Monte Carlo, Metropolis-Hastings Algorithm

Lecture 8. Continuation of topics from Lecture 7. Hidden Markov Models: See also tutorial article by L. R. Rabiner, Proc. IEEE, Vol. 77 1989, 257--286.

Lecture 9. Hidden Markov Models (cont.): Examples, main problems, forward algorithm.

Lecture 10. Hidden Markov Models (cont.): Viterbi algorithm, Profile HMMs

Lecture 11 (courtesy of Ben Gillen). Profile HMMs (cont.), Viterbi for Profile HMMs. Parameter estimation.

Lecture 12. Parameter estimation (cont.): Baum-Welch, EM algorithm

Lecture 13. Continuous Time Markov Chains: Definition, Examples

Lecture 14. Continuous Time Markov Chains (cont.): Long run behavior

Lecture 15. Martingales: Conditional Expectation, Definition

Lecture 16. Martingales (cont.): Examples

Lecture 17. Martingales (cont.): Optional Stopping Theorem.

Lecture 18. Brownian Motion

Lecture 19. Brownian motion (cont.)

Last updated April 1, 2007.