Stochastic Processes Course

Lesson 1 • Expectation and Moments

Defines Lebesgue integration-based expectation and moment functionals. These tools quantify average behaviour and variability in stochastic models.

Lesson 2 • Probability Measures and Axioms

Introduces Kolmogorov axioms and measure-theoretic probability. Provides the rigorous foundation for all probabilistic statements in later chapters.

Lesson 3 • Convergence Concepts in Probability

Distinguishes almost-sure, in-probability, and mean-square convergence. Prepares students to analyse limiting behaviour of stochastic sequences.

Lesson 4 • Random Variables and Distributions

Covers discrete and continuous random variables and their distributions. Links abstract measure theory to practical probabilistic modelling.

Lesson 1 • Definition and Classification of Processes

Formalises a stochastic process as an indexed family of random variables. Classification by discrete or continuous time and state space guides model selection.

Lesson 2 • Stationarity and Ergodicity

Defines strict and wide-sense stationarity and ergodic properties. These concepts underpin time-series analysis and long-run average computations.

Lesson 3 • Finite-Dimensional Distributions

Characterises processes through their finite-dimensional distribution families. Kolmogorov's consistency theorem guarantees existence of well-defined processes.

Lesson 1 • State Classification

Classifies states as transient, recurrent, or absorbing based on return probabilities. Classification determines long-run occupancy and absorption outcomes.

Lesson 2 • Absorption and First-Passage Analysis

Computes absorption probabilities and expected hitting times using linear systems. These calculations are essential for reliability and queueing applications.

Lesson 3 • Stationary and Limiting Distributions

Derives conditions for existence and uniqueness of stationary distributions. Connects ergodic theory to long-run frequency interpretations.

Lesson 4 • Reversibility and Detailed Balance

Introduces time-reversible chains and the detailed balance condition. Reversibility simplifies stationary distribution computation in complex networks.

Lesson 1 • Poisson Process Definition and Properties

Characterises the Poisson process via independent increments and Poisson-distributed counts. Three equivalent definitions are compared for modelling flexibility.

Lesson 2 • Renewal Reward Theorem and Applications

Applies the renewal reward theorem to compute long-run average rewards per unit time. Covers inventory, maintenance, and replacement policy optimisation.

Lesson 3 • Renewal Theory Fundamentals

Defines renewal processes and derives the renewal equation and elementary renewal theorem. Provides the asymptotic framework for long-run rate analysis.

Lesson 4 • Superposition and Thinning

Derives rules for merging and splitting Poisson processes. These operations model routing, multiplexing, and random sampling of event streams.

Lesson 1 • Stationary Distributions and Ergodicity

Extends discrete-time ergodic theory to continuous-time settings. Conditions for positive recurrence and unique stationary distributions are established.

Lesson 2 • Generator Matrix and Kolmogorov Equations

Defines the infinitesimal generator and derives forward and backward equations. These differential equations govern the evolution of transition probabilities.

Lesson 3 • Exponential Holding Times and Rates

Establishes that memoryless sojourn times follow exponential distributions. This property enables the generator-based formulation of continuous-time chains.

Lesson 4 • Birth-Death Processes

Analysis chains with transitions only to adjacent states, covering population and queue models. Closed-form stationary distributions are derived via detailed balance.

Lesson 1 • Martingale Definition and Examples

Defines martingales, submartingales, and supermartingales with canonical examples. Recognizing martingale structure enables powerful analytical shortcuts.

Lesson 2 • Conditional Expectation and Filtrations

Defines conditional expectation rigorously and introduces filtrations as information flows. These concepts underpin the formal definition of martingales.

Lesson 3 • Stopping Times and Optional Stopping Theorem

Defines stopping times and states conditions for the optional stopping theorem. Applies the theorem to compute expected exit times and ruin probabilities.

Lesson 4 • Martingale Convergence Theorems

Presents Doob's upcrossing inequality and almost-sure convergence theorem. These results establish when martingale limits exist and are integrable.

Lesson 1 • Markov and Martingale Properties

Proves Brownian motion is both a Markov process and a martingale. Derives key martingales including the exponential and quadratic Brownian martingales.

Lesson 2 • Gaussian Processes and Covariance Kernels

Defines Gaussian processes through their finite-dimensional Gaussian distributions. Covariance kernel selection determines smoothness and correlation structure.

Lesson 3 • Construction of Brownian Motion

Builds Brownian motion via the Wiener measure and Lévy-Ciesielski construction. Establishes existence of a process with continuous paths and independent increments.

Lesson 4 • Properties of Brownian Sample Paths

Analyzes quadratic variation, Hölder continuity, and the law of the iterated logarithm. These path properties are essential for stochastic calculus development.

Lesson 1 • Girsanov's Theorem and Change of Measure

States Girsanov's theorem for changing drift via an equivalent probability measure. This result is fundamental to risk-neutral pricing and likelihood ratio methods.

Lesson 2 • Numerical Methods for SDEs

Introduces Euler-Maruyama and Milstein schemes for simulating SDE solutions. Analyzes strong and weak convergence orders for practical error control.

Lesson 3 • Stochastic Differential Equations

Formulates SDEs and establishes existence and uniqueness under Lipschitz conditions. Covers linear SDEs and the Ornstein-Uhlenbeck process as canonical examples.

Lesson 4 • Ito's Formula and Change of Variables

Derives Ito's formula as the stochastic chain rule for twice-differentiable functions. Applies the formula to compute dynamics of transformed processes.