Advanced Probability and Statistical Methods Course

Lesson 1 • Axioms and Properties of Probability

Introduces Kolmogorov axioms and derives key properties. Provides the logical foundation for all probability calculations in the course.

Lesson 2 • Combinatorics for Probability

Covers counting techniques essential for computing probabilities in finite spaces. Bridges combinatorial reasoning with probability model construction.

Lesson 3 • Conditional Probability and Independence

Defines conditional probability and statistical independence formally. Connects these concepts to real-world reasoning about dependent events.

Lesson 4 • Sample Spaces and Event Algebra

Defines sample spaces, sigma-algebras, and event operations. Establishes the set-theoretic language used throughout all subsequent probability work.

Lesson 1 • Generating Functions and Characteristic Functions

Introduces probability-generating, moment-generating, and characteristic functions. Demonstrates their use in identifying distributions and proving limit theorems.

Lesson 2 • Discrete Random Variables

Defines discrete random variables through probability mass functions and CDFs. Introduces Bernoulli, binomial, geometric, and Poisson distributions.

Lesson 3 • Expectation, Variance, and Moments

Derives expectation and variance operators and their algebraic properties. Introduces moment-generating functions as tools for characterising distributions.

Lesson 4 • Continuous Random Variables

Introduces probability density functions and continuous CDFs. Covers uniform, exponential, and normal distributions with their key properties.

Lesson 1 • Conditional Distributions and Expectation

Derives conditional distributions and the conditional expectation operator. Introduces the tower property as a key tool for computing expectations.

Lesson 2 • Multivariate Normal Distribution

Characterises the multivariate normal via its mean vector and covariance matrix. Derives marginal and conditional distributions and the affine transformation property.

Lesson 3 • Copulas and Dependence Modelling

Introduces Sklar's theorem and copula families for modelling non-linear dependence. Applies copulas to joint tail behaviour and risk aggregation.

Lesson 4 • Joint and Marginal Distributions

Defines joint PMFs and PDFs and derives marginal distributions by integration or summation. Establishes the multivariate framework for all subsequent chapters.

Lesson 1 • Concentration Inequalities

Introduces Markov, Chebyshev, Chernoff, and Hoeffding inequalities for tail probability bounds. Applies these tools to sample size determination and algorithm analysis.

Lesson 2 • Laws of Large Numbers

Proves the weak and strong laws of large numbers under standard conditions. Connects these results to the justification of frequentist probability and simulation.

Lesson 3 • Delta Method and Variance Stabilisation

Derives the delta method for asymptotic distributions of smooth functions of estimators. Applies variance-stabilising transformations to improve normal approximations.

Lesson 4 • Modes of Stochastic Convergence

Defines convergence in probability, almost surely, in mean, and in distribution. Clarifies the hierarchy of convergence modes with examples and counterexamples.

Lesson 1 • Method of Moments and Bayesian Estimation

Introduces method-of-moments estimation and contrasts it with Bayesian posterior estimation. Covers conjugate priors and posterior mean as a Bayes estimator.

Lesson 2 • Properties of Point Estimators

Defines unbiasedness, consistency, efficiency, and sufficiency for point estimators. Provides criteria for comparing competing estimators in applied settings.

Lesson 3 • Confidence Intervals and Pivotal Quantities

Constructs exact and approximate confidence intervals using pivotal quantities. Interprets coverage probability and distinguishes confidence from credible intervals.

Lesson 4 • Cramér-Rao Lower Bound

Derives the CRLB as the minimum variance for unbiased estimators. Identifies efficient estimators and connects the bound to Fisher information.

Lesson 1 • Generalised Likelihood Ratio Tests

Derives the GLRT for composite hypotheses and establishes Wilks' theorem for asymptotic chi-squared distribution. Applies GLRT to nested model comparison.

Lesson 2 • Statistical Decision Theory

Frames estimation and testing as decision problems with loss functions and risk. Introduces admissibility, minimax rules, and Bayes decision procedures.

Lesson 3 • Multiple Testing and Error Rate Control

Addresses the multiple comparisons problem and introduces FWER and FDR control procedures. Applies Bonferroni, Holm, and Benjamini-Hochberg corrections.

Lesson 4 • Foundations of Hypothesis Testing

Defines null and alternative hypotheses, Type I and II errors, and power. Establishes the Neyman-Pearson framework as the basis for optimal test construction.

Lesson 1 • Mixed Effects and Hierarchical Models

Introduces random effects to account for clustered and longitudinal data structures. Derives REML estimation and interprets fixed vs. random effect components.

Lesson 2 • Regression Diagnostics and Assumption Checking

Identifies violations of linearity, homoscedasticity, normality, and independence assumptions. Applies residual plots, influence measures, and formal diagnostic tests.

Lesson 3 • Simple and Multiple Linear Regression

Derives OLS estimators via matrix algebra and establishes the Gauss-Markov theorem. Covers inference on coefficients, model fit, and prediction intervals.

Lesson 4 • Variable Selection and Regularisation

Covers subset selection, AIC/BIC criteria, and shrinkage methods for high-dimensional regression. Derives ridge and lasso estimators and their bias-variance tradeoffs.

Lesson 1 • Bayesian Model Specification

Covers prior elicitation strategies including informative, weakly informative, and non-informative priors. Connects prior choice to posterior sensitivity and model identifiability.

Lesson 2 • Posterior Computation and Conjugacy

Derives closed-form posteriors for conjugate models and identifies their limitations. Motivates numerical methods for non-conjugate posterior computation.

Lesson 3 • Markov Chain Monte Carlo Methods

Derives Metropolis-Hastings and Gibbs sampling algorithms for posterior simulation. Covers convergence diagnostics and effective sample size assessment.

Lesson 4 • Bayesian Model Comparison

Introduces Bayes factors, marginal likelihoods, and information criteria for model selection. Applies WAIC and LOO-CV for predictive model comparison.