MATH 285: STOCHASTIC PROCESSES (SPRING 2007)

Copyright c2007, R. J. Williams

DO NOT REPRODUCE WITHOUT PERMISSION

Please send notice of typos to williams at math dot ucsd dot edu

The start of a new main topic is indicated by highlighting the topic in red.

  • Lecture 1. Introduction to Stochastic Processes
  • Lecture 2. Law of a Stochastic Process and Discrete Time Markov Chains
  • Lecture 3. Examples and classification of states.
  • Lecture 4. Hitting times and absorption probabilities.
  • Lecture 5. Hitting times and absorption probabilities - random walk example.
  • Lecture 6. Long Run Behavior of Markov Chains.
  • Lecture 7. Periodicity, transience, recurrence, basic limit theorem.
  • Lecture 8. Sketch of proofs related to long run behavior.
  • Lecture 9. Reversible Markov chains. See F. P. Kelly, Reversibility and Stochastic Networks for more on reversible Markov chains.
  • Lecture 10. Hidden Markov Models
    See tutorial article by L. R. Rabiner, Proc. IEEE, Vol. 77 1989, 257--286.
  • Lecture 11. Algorithms for computing quantities associated with HMMs.
  • Lecture 12. Continuous Time Markov Chains.
  • Lecture 13. Examples: Poisson process, birth-death processes.
  • Lecture 14. Long run behavior of continuous time Markov chains.
  • Lecture 15. Conditional Expectation and Martingales.
  • Lecture 16. Conditional Expectation and Martingales (continued).
  • Lecture 17. Examples of Martingales.
  • Lecture 18. Brownian Motion.
  • Lecture 19. Brownian Motion: a Gaussian process, continuous time martingale and Markov process.

    Last updated April 1, 2007.