Machine Learning Classification Algorithms Course

Lesson 1 • Data Representation and Preprocessing

Covers encoding, scaling, and handling missing values to prepare datasets for classifiers. Directly impacts model accuracy and is required before any algorithm is applied.

Lesson 2 • The Machine Learning Workflow

Walks through the end-to-end pipeline from raw data to deployed model. Provides a repeatable framework students apply in every subsequent chapter.

Lesson 3 • Setting Up the Development Environment

Configures Python-based tooling including key libraries for data manipulation and modelling. Gives students a working environment to run all course exercises immediately.

Lesson 4 • Mathematical Prerequisites for Classification

Reviews probability, linear algebra, and calculus concepts essential for understanding classifiers. Ensures students can follow algorithm derivations in later chapters.

Lesson 1 • The Confusion Matrix Explained

Introduces the confusion matrix as the foundation for all classification metrics. Connects raw prediction counts to actionable performance insights.

Lesson 2 • Bias, Variance, and Overfitting

Explains the bias-variance trade-off and its effect on generalisation. Equips students to diagnose underfitting and overfitting before tuning any algorithm.

Lesson 3 • Threshold-Based and Ranking Metrics

Covers ROC curves, AUC, and precision-recall curves for probability-outputting classifiers. Extends metric knowledge to scenarios requiring threshold tuning.

Lesson 4 • Cross-Validation Strategies

Teaches k-fold, stratified, and leave-one-out cross-validation to produce reliable performance estimates. Prevents overfitting to a single train-test split.

Lesson 1 • Maximum Likelihood and Log-Loss

Derives the log-loss objective from maximum likelihood estimation. Connects the maths to the optimisation process students will apply in training.

Lesson 2 • Multiclass Logistic Regression

Extends binary logistic regression to multiclass problems using OvR and softmax strategies. Prepares students for datasets with more than two target classes.

Lesson 3 • From Linear Regression to Logistic Regression

Shows why linear regression fails for classification and how the sigmoid function fixes it. Establishes the probabilistic interpretation central to logistic regression.

Lesson 4 • Interpreting and Deploying Logistic Models

Covers coefficient interpretation, odds ratios, and exporting trained models. Bridges model building to practical deployment and stakeholder communication.

Lesson 1 • Rule Extraction and Interpretability

Shows how to extract human-readable if-then rules from trained trees. Connects tree outputs to business logic and compliance requirements.

Lesson 2 • Feature Importance from Trees

Explains how trees compute feature importance scores based on impurity reduction. Gives students a built-in tool for feature selection and model explanation.

Lesson 3 • Splitting Criteria: Gini and Entropy

Derives Gini impurity and information gain as measures for choosing the best split. Explains how these criteria drive the learning algorithm at each node.

Lesson 4 • Controlling Overfitting in Trees

Covers pre-pruning hyperparameters and post-pruning techniques to reduce overfitting. Applies bias-variance concepts from Chapter 2 to tree-specific controls.

Lesson 1 • Stacking and Blending Ensembles

Teaches meta-learning by training a second-level model on base classifier outputs. Extends ensemble knowledge to flexible architectures that maximise predictive power.

Lesson 2 • XGBoost and LightGBM

Introduces optimised gradient boosting libraries with regularisation and speed improvements. Prepares students for competition-grade and production-grade classification tasks.

Lesson 3 • Gradient Boosting Machines

Derives gradient boosting as sequential residual fitting using gradient descent in function space. Explains the core algorithm behind modern high-performance classifiers.

Lesson 4 • Random Forests in Depth

Covers bootstrap aggregation, random feature subsets, and majority voting in Random Forests. Builds directly on decision tree knowledge from Chapter 4.

Lesson 1 • Tuning SVMs with Grid and Random Search

Applies hyperparameter search strategies to optimise C and gamma for SVM classifiers. Reinforces cross-validation skills from Chapter 2 in an SVM context.

Lesson 2 • Soft-Margin SVM and the C Parameter

Introduces slack variables to handle non-separable data and the C regularisation parameter. Connects SVM tuning to the bias-variance trade-off from Chapter 2.

Lesson 3 • SVMs for Multiclass and Imbalanced Data

Extends SVMs to multiclass settings and addresses class imbalance with class weights. Prepares students to apply SVMs to realistic, unbalanced classification problems.

Lesson 4 • Maximum Margin Classification

Derives the hard-margin SVM objective and explains support vectors geometrically. Provides the intuition needed to understand soft-margin and kernel extensions.

Lesson 1 • Comparing Probabilistic and Instance-Based Models

Benchmarks Naive Bayes and KNN against logistic regression and tree-based models. Guides students in selecting the right classifier for a given problem profile.

Lesson 2 • Scaling and Efficiency in KNN

Addresses the computational cost of KNN and techniques to accelerate neighbour search. Ensures students can apply KNN to datasets of practical size.

Lesson 3 • K-Nearest Neighbors Algorithm

Explains the KNN decision rule, distance metrics, and the effect of k on the boundary. Provides a non-parametric baseline that requires no explicit training phase.

Lesson 4 • Naive Bayes for Text Classification

Applies Multinomial Naive Bayes to bag-of-words and TF-IDF representations. Demonstrates a practical, high-speed classifier for natural language tasks.

Lesson 1 • Building Scikit-Learn Pipelines

Encapsulates preprocessing and modelling steps into a single pipeline object. Prevents data leakage and simplifies cross-validation and deployment workflows.

Lesson 2 • Automated Machine Learning Overview

Introduces AutoML tools that automate algorithm selection and hyperparameter search. Positions automation as a productivity tool, not a replacement for understanding.

Lesson 3 • Statistical Model Comparison

Applies statistical tests to determine whether performance differences between models are significant. Prevents selecting a model based on noise rather than genuine improvement.

Lesson 4 • Final Model Selection and Documentation

Establishes a decision framework for choosing the production model and documenting choices. Prepares students to justify model selection to technical and business stakeholders.