Principal Component Analysis Course

Lesson 1 • Review of Descriptive Statistics

Covers mean, variance, and standard deviation as building blocks for PCA. Connects univariate summaries to multivariate analysis needs.

Lesson 2 • Eigenvalues and Eigenvectors Primer

Defines eigenvalues and eigenvectors and explains their geometric meaning. Directly prepares students for understanding principal components as eigenvectors.

Lesson 3 • Covariance and Correlation Concepts

Explains how covariance and correlation quantify linear relationships between variables. Establishes the statistical basis for the covariance matrix used in PCA.

Lesson 4 • Understanding High-Dimensional Data

Introduces the curse of dimensionality and why reducing dimensions matters. Sets the motivation for PCA as a practical solution to data complexity.

Lesson 1 • Data Centering and Scaling

Explains why centering is compulsory and when scaling is necessary before PCA. Demonstrates how unscaled variables with large ranges dominate principal components.

Lesson 2 • Constructing the Covariance Matrix

Walks through the step-by-step computation of a covariance matrix from raw data. Connects the matrix structure to pairwise variable relationships established earlier.

Lesson 3 • Correlation Matrix as an Alternative

Contrasts the correlation matrix with the covariance matrix and identifies when each is preferred. Reinforces the impact of variable scaling on PCA outcomes.

Lesson 1 • Loadings and Component Interpretation

Defines loadings as eigenvector coefficients and explains how they relate variables to components. Teaches systematic interpretation of what each component represents.

Lesson 2 • Eigendecomposition of the Covariance Matrix

Applies eigendecomposition to the covariance matrix to extract principal directions. Links eigenvalues to explained variance and eigenvectors to component axes.

Lesson 3 • Singular Value Decomposition Approach

Introduces SVD as a numerically stable alternative to eigendecomposition for PCA. Explains the relationship between singular values and eigenvalues of the covariance matrix.

Lesson 4 • Computing Principal Component Scores

Projects original data onto principal component axes to produce scores. Demonstrates how scores represent observations in the new reduced coordinate system.

Lesson 1 • Information Loss and Reconstruction Error

Quantifies the trade-off between dimensionality reduction and information loss using reconstruction error. Connects retained variance to practical accuracy requirements.

Lesson 2 • Proportion of Variance Explained

Calculates the proportion of total variance captured by each component using eigenvalue ratios. Establishes the quantitative basis for component selection decisions.

Lesson 3 • Kaiser Criterion and Other Rules

Presents the Kaiser rule and parallel analysis as formal component retention criteria. Compares their assumptions and applicability across different dataset types.

Lesson 4 • Choosing the Final Number of Components

Synthesises all selection criteria into a decision framework for choosing the number of components. Emphasises aligning the choice with the downstream analytical goal.

Lesson 1 • Visualising PCA Results

Creates scree plots, score scatter plots, and biplots using matplotlib and seaborn. Visualisation connects numerical outputs to interpretable insights.

Lesson 2 • Data Preparation in Python

Covers loading, cleaning, and scaling datasets using pandas and scikit-learn. Prepares students to handle real data before applying PCA programmatically.

Lesson 3 • Implementing PCA from Scratch

Builds PCA using NumPy to reinforce understanding of the underlying algorithm. Comparing scratch implementation to scikit-learn validates conceptual mastery.

Lesson 4 • Building a Reusable PCA Pipeline

Encapsulates preprocessing and PCA into a scikit-learn Pipeline for reproducibility. Introduces best practices for saving and loading fitted PCA models.

Lesson 1 • PCA in Classification Pipelines

Integrates PCA into classification workflows and evaluates accuracy trade-offs. Highlights cases where PCA improves and where it harms classifier performance.

Lesson 2 • PCA for Feature Reduction Before Modeling

Uses PCA to reduce input features before training supervised models. Demonstrates how fewer components can reduce overfitting and training time.

Lesson 3 • Anomaly Detection with PCA

Uses reconstruction error from PCA to identify anomalous observations. Establishes a threshold-based anomaly detection framework using PCA residuals.

Lesson 4 • PCA for Clustering Enhancement

Applies PCA before k-means and hierarchical clustering to improve cluster separation. Explains why high-dimensional clustering benefits from prior dimensionality reduction.

Lesson 1 • Kernel PCA for Nonlinear Data

Extends PCA to nonlinear structures using the kernel trick. Demonstrates how kernel PCA captures variance that standard PCA misses in curved manifolds.

Lesson 2 • Sparse PCA for Interpretable Components

Introduces sparse PCA, which enforces zero loadings to produce interpretable components. Contrasts sparsity-regularised components with dense standard PCA loadings.

Lesson 3 • Probabilistic PCA and Bayesian Extensions

Frames PCA as a latent variable model to enable probabilistic inference and missing data handling. Introduces Bayesian PCA for automatic component selection.

Lesson 4 • Robust PCA for Outlier Resistance

Addresses PCA sensitivity to outliers using robust estimation techniques. Decomposes data into low-rank and sparse components to isolate corrupted observations.

Lesson 1 • Cross-Validation for PCA

Applies cross-validation to assess the stability of component structure across data splits. Prevents overfitting the number of components to a single dataset.

Lesson 2 • Reproducibility and Documentation Standards

Establishes standards for documenting PCA workflows to ensure reproducibility. Covers version control, parameter logging, and analysis reporting conventions.

Lesson 3 • Communicating PCA Results

Translates PCA outputs into clear narratives for technical and non-technical audiences. Covers effective use of tables, plots, and plain-language summaries.

Lesson 4 • Validating PCA Assumptions

Tests linearity, multivariate normality, and sample size adequacy before applying PCA. Identifies when assumption violations require alternative methods.