Math 285 Stochastic processes

Winter 2022: MWF 4:00-4:50pm, APM 5402


Course Information

Instructional Staff and Office Hours

Name Role Office E-mail Zoom link
Yuriy Nemish  Instructor   APM 6442 link

Lectures   Monday, Wednesday, Friday  4:00 - 4:50 PM
Office hours   Thursday  1:30 - 3:00 PM


Welcome to Math 285: a one quarter course in stochastic processes. According to the UC San Diego Course Catalog, the topics covered are Markov chains, hidden Markov models, martingales, Brownian motion, Gaussian processes.

Here is a more detailed listing of course topics, in the sequence they will be covered, together with the relevant section(s) of the textbook. While each topic corresponds to approximately one lecture, there will be some give and take here. This is a rough schedule that will be updated during the term. (TBA)

Lecture Week Topic Lawler Slides Videos Preliminary Slides
1/31 Introduction. Definition of Markov processes  § 1.1Slides 1Slides 1
1/51 Transition probabilities. Hitting times  -Slides 2Slides 2
1/71 Hitting times. First step analysis  -Slides 3Lecture 3Slides 3
1/102 First step analysis. Stopping times  -Slides 4Lecture 4Slides 4
1/122 Irreducible Markov chains. Random walks of graphs  -Slides 5Lecture 5Slides 5
1/142 Stationary distributions. Long-run behaviour  § 1.2Slides 6Lecture 6Slides 6
1/193 Periodic, aperiodic, reducible, irreducible Markov chains  § 1.3Slides 7Lecture 7Slides 7
1/213 Markov chains with finite state space  §§ 1.3, 1.5Slides 8Lecture 8Slides 8
1/244 Markov chains with infinite state space  §§ 1.3, 2.3Slides 9Lecture 9Slides 9
1/264 Positive and null recurrence  §§ 2.3Slides 10Lecture 10Slides 10
1/284 Positive and null recurrence  §§ -Slides 11Lecture 11Slides 11
1/315 Ergodic theorem  §§ -Slides 12Lecture 12Slides 12
2/25 Time reversal  §§ 7.1Slides 13Lecture 13Slides 13
2/45 MCMC  §§ 7.3Slides 14Lecture 14Slides 14
2/76 Branching processes  §§ 2.4Slides 15Lecture 15Slides 15
2/96 Hidden Markov Models  §§ -Slides 16Lecture 16Slides 16
2/116 Hidden Markov Models. Viterbi algorithm  §§ -Slides 17Lecture 17Slides 17
2/147 Continuous-time Markov chains  §§ 3.1-3.2Slides 18Lecture 18Slides 18
2/167 Strong Markov property. Embedded jump chain. Infinitesimal description  §§ 3.1-3.4Slides 19Lecture 19Slides 19
2/187 Kolmogorov's equations. Poisson processes  §§ 3.1-3.4Slides 20Lecture 20Slides 20
2/238 Poisson processes. Birth and death chains  §§ 3.1-3.4Slides 21Lecture 21Slides 21
2/258 Recurrence and transience. Stationary distributions  §§ 3.1-3.4Slides 22Lecture 22Slides 22
2/289 Long-run behavior of continuous-time MC. Conditional expectation  §§ 5.1Slides 23Lecture 23Slides 23
3/29 Martingales  §§ 5.2-5.3Slides 24Lecture 24Slides 24
3/49 Martingales. Doob's maximal inequality  §§ 5.6Slides 25Lecture 25Slides 25
3/710 Martingale convergence theorem  §§ 5.5Slides 26Lecture 26Slides 26

Lecture:  You are responsible for material presented in the lecture whether or not it is discussed in the textbook. You should expect homework questions that will test your understanding of concepts discussed in the lecture.

Homework:  Homework assignments are posted below, and will be due at 11:59 PM on the indicated due date.  You must turn in your homework through Gradescope; if you have produced it on paper, you can scan it or simply take clear photos of it to upload. Your lowest homework score will be dropped.  It is allowed and even encouraged to discuss homework problems with your classmates and your instructor, but your final write up of your homework solutions must be your own work.

Evaluation:  There will be no exams in this course. Your grade will be determined by your homework.

Administrative Links:    Here are two links regarding UC San Diego policies on exams:

Regrade Policy:  


Your course grade will be based on the following scale:

A+ A A- B+ B B- C+ C C-
97 93 90 87 83 80 77 73 70

The above scale is guaranteed: for example, if your cumulative average is 80, your final grade will be at least B-. However, your instructor may adjust the above scale to be more generous.

Academic Integrity:  UC San Diego code of academic integrity outlines the expected academic honesty of all studentd and faculty, and details the consequences for academic dishonesty. The main issues are cheating and plagiarism, of course, for which we have a zero-tolerance policy. (Penalties for these offenses always include assignment of a failing grade in the course, and usually involve an administrative penalty, such as suspension or expulsion, as well.) However, academic integrity also includes things like giving credit where credit is due (listing your collaborators on homework assignments, noting books or papers containing information you used in solutions, etc.), and treating your peers respectfully in class. In addition, here are a few of our expectations for etiquette in and out of class.


Students requesting accommodations for this course due to a disability must provide a current Authorization for Accommodation (AFA) letter issued by the Office for Students with Disabilities (OSD) which is located in University Center 202 behind Center Hall. The AFA letter may be issued by the OSD electronically or in hard-copy; in either case, please make arrangements to discuss your accommodations with me in advance (by the end of Week 2, if possible). We will make every effort to arrange for whatever accommodations are stipulated by the OSD. For more information, see here.



Weekly homework assignments are posted here. Homework is due by 11:59 PM on the posted date, through Gradescope. Late homework will not be accepted.