Math 4740: Stochastic Processes

Spring 2013

MWF 9:05-9:55 in 251 Malott Hall


Course home page (this page):

For many years this course was taught by longtime Cornellian Rick Durrett (now at Duke Univeristy) who also wrote our textbook. This year's course is modeled on the one taught last year by Nate Eldredge.

New: practice problems for the final

The final is on Thursday May 16 from 9-11:30 in Malott 406.

New: Office hours during exam period:


Section numbers on the right refer to the (online only) 2nd edition of Durrett.

date due topic reference
Jan 21 Random walk, central limit theorem. Why randomness? (Durrett 2nd ed.)
Jan 23 Markov property. Multi-step transition probabilities. Examples 1.1-1.2
Jan 25 Recurrence and transience 1.3, worksheet 1
Jan 28 Strong Markov property. Expected number of visits E_x N(y) 1.3
Jan 30 Closed and irreducible sets. Criteria for recurrence 1.3
Feb 1 pset 1 Coin flipping example. Intro to stationary distribution. 1.4, worksheet 2
Feb 4 Stationary distribution: 2x2 case, and what can go wrong with convergence 1.4, Sage
Feb 6 Doubly stochastic chains. Detailed balance, random walk on a graph 1.6
Feb 8 pset 2 Existence of stationary distribution; expected return time 1.5, worksheet 3
Feb 11 Irreducibility, uniqueness of stationary distribution 1.5
Feb 13 Aperiodicity, convergence to equilibrium 1.5
Feb 15 pset 3 English language as a Markov chain worksheet 4
Feb 18 Proof of the convergence theorem 1.7
Feb 20 Ergodic theorem. Exit distributions 1.7(Thm 1.23), 1.8
Feb 22 pset 4 Exit times 1.9, worksheet 5
Feb 25 Infinite state space 1.10
Feb 27 Branching processes 1.10(Ex 1.52)
Mar 1 ***Prelim*** covering finite Markov chains (all topics through Feb 22) 1.1-1.9
Mar 4 Pagerank papers
Mar 6 Markov chain Monte Carlo (MCMC) MCMT chapter 3
Mar 8 pset 5 Exponential distribution, lack of memory 2.1
Mar 11 Poisson distribution, Poisson process 2.2
Mar 13 Poisson approximation to the binomial 2.2
Mar 15 pset 6 Compound Poisson processes 2.3, worksheet 6
Mar 18-22 ***spring break***
Mar 25 Sums with a random number of terms 2.3
Mar 27 Thinning and conditioning 2.4
Mar 29 pset 7 Higher dimensional Poisson processes references
Apr 1 Point processes. Intro to continuous time Markov chains 4.1
Apr 3 Yule process. Constructing a chain from its jump rates 4.1
Apr 5 pset 8 Kolmogorov differential equation 4.2, worksheet 7
Apr 8 Stationary distribution solves pi Q = 0 4.3
Apr 10 Detailed balance, continuous time random walk 4.3, worksheet 8
Apr 12 project Convergence theorem. Hitting and exit distributions 4.4
Apr 15 Conditional expectation 5.1
Apr 17 Martingales 5.2
Apr 19 pset 9 You can't beat an unfavorable game 5.3
Apr 22 Optional stopping 5.3,worksheet 9
Apr 24 Applications of optional stopping. Uncorrelated increments 5.4,worksheet 10
Apr 26 project Martingale convergence theorem 5.5
Apr 29 Martingales in prediction markets paper
May 1 Brownian motion: definition, construction, reflection principle 1st ed. 6.1-6.3
May 3 pset 10 Brownian motion: distribution of the maximum, dimension of the zero set BM book

Prerequisite: An introductory probability course such as MATH 4710, BTRY 4080, ORIE 3600, ECON 3190. General proficiency in calculus and linear algebra. If unsure about your preparation, please discuss it with me.


Essentials of Stochastic Processes, by Durrett.

The first edition, made out of dead trees, is available at the Cornell store, and a copy is on reserve at the Mathematics Library (4th floor Malott Hall).

Here is a list of known typos in the first edition.

And here is the second edition, made out of electrons. We are compiling a list of typos in the 2nd edition.

Other sources

If Durrett crimps your style, take a look at these books which cover roughly the same material:

For the extra topics, we may dip into a couple more advanced books:

Problem Sets

Problem Set 1, due Friday, February 1.

Problem Set 2, due Friday, February 8. Solutions

Problem Set 3, due Friday, February 15. Solutions

Problem Set 4, due Friday, February 22. Solutions

Problem Set 5, due Friday, March 8. Solutions

Problem Set 6, due Friday, March 15. Solutions

Problem Set 7, due Friday, March 29. Solutions

Problem Set 8, due Friday, April 5. Solutions

Problem Set 9, due Friday, April 19. Solutions

Problem Set 10, due Friday, May 3. Solutions

Please check out the problem set guidelines for how to write a good solution.

You are strongly encouraged to type your problem sets using LaTeX (or LyX). LaTeX is very versatile and widely used for writing technical documents of all kinds. It will serve you well if you go on in math or another technical field. Besides, you will need it for your project! Neatly handwritten problem sets are also acceptable.

New: Here is a sample LaTeX file and the pdf output it generates.

Here is a video tutorial to get you started with LaTeX. Another good resource is the LaTeX wikibook. The Cornell math support center can help you get LaTeX up and running. (Ask for a math major tutor!)

Two times during the semester, you may have an automatic 72-hour extension on a problem set. This does not require prior approval, just hand it in at the beginning of the next class with a note saying you are using an extension. Otherwise, late homework will not be accepted except in an emergency (in which case you must inform me as soon as possible).

Group work policy: Working together with other students to solve the problems is encouraged. However, you must understand the solution yourself and write it up entirely in your own words. Discussing your solution with others is fine, but do not share your written solution with another student. If you work in a group, list the group members at the top of your problem set.


One in-class prelim on Friday, March 1. Final exam on Thursday, May 16. The exams are closed book.

If a conflict prevents you from completing the exams on these dates, please let me know immediately.

Practice problems for the prelim.

Solutions to the prelim. The mean was 42 out of 60 and the standard deviation was 9. Approximate grade ranges: A (47-57), B (31-46), C (22-30).

Course Project

The course project allows you to explore a topic of your choice in depth. You will choose a peer-reviewed journal article that uses stochastic processes to model some real world phenomenon, and write a critical summary of the article analyzing the strengths and weaknesses of the model it proposes.

Check out the project guidelines for details.


Graduate students: If you are taking the course for credit, you will need to complete all the regular coursework (homework, exams and project), and will be assigned a grade like any other student. If you are taking the course S/U, you still need to do the coursework; I will compute a letter grade as usual and then assign you an S if the letter grade is at least C-. If you are not interested in doing the coursework, you are welcome to audit the course instead.

Laptop-free zone

Recent experiments show that even though people feel like they are accomplishing more when they multitask, they actually accomplish less. Higher level mathematics really demands your full attention. It's hard enough to learn stochastic processes without the temptation of facebook and xkcd (yes, the lure of xkcd is strong. very strong.) So please keep laptops, ipads, phones and other gizmos in your bag during class. If you really must use a device during class (for example, to take notes on your tablet or laptop), you may do so in the back row where it does not distract anyone.

Students with disabilities

The Cornell disability policy is as follows. If you have a disability‚Äźrelated need for reasonable academic adjustments in this course, provide me with an accommodation letter from Student Disability Services. Students are expected to give two weeks notice of the need for accommodations. If you need immediate accommodations, please arrange to meet with me within the first two class meetings.

Academic integrity

Please read carefully the group work policy for problem sets. I, your classmates, and the entire Cornell community expect you to complete this course based on your own work, to abide by relevant rules for coursework and exams, and to clearly attribute any use of the work of others. The Cornell Academic Integrity Policy spells out the nasty details of what happens otherwise.

Back to Lionel Levine's home page