Speech Recognition Course

Lesson 1 • Phonemes and Linguistic Units

Introduces phonemes, syllables, words, and prosody as linguistic units. Provides the vocabulary used throughout all subsequent recognition chapters.

Lesson 2 • Acoustic Properties of Speech

Examines frequency, amplitude, and time-domain characteristics of speech signals. Connects physical production to measurable acoustic features.

Lesson 3 • How Humans Produce Speech

Covers the anatomy of the vocal tract and articulatory phonetics. Establishes the physical basis that all recognition models must account for.

Lesson 1 • Noise and Pre-Processing

Covers pre-emphasis, voice activity detection, and noise reduction. Ensures feature quality before feeding data into any recognition model.

Lesson 2 • Frequency-Domain Analysis

Applies the Fourier transform to reveal spectral content of speech. Connects frequency analysis to feature extraction used in later chapters.

Lesson 3 • MFCC Feature Extraction

Derives Mel-Frequency Cepstral Coefficients step by step. MFCCs are the dominant hand-crafted features used in classical recognition systems.

Lesson 4 • Filter Banks and Mel Scale

Introduces perceptual frequency scales and filter bank design. Explains why mel-scale filters align with human auditory perception.

Lesson 1 • Introduction to Statistical Modeling

Frames speech recognition as a probabilistic decoding problem. Establishes Bayes' theorem and likelihood estimation as the theoretical backbone.

Lesson 2 • Gaussian Mixture Models for Acoustics

Introduces GMMs as emission distributions within HMMs. Explains how mixtures capture the variability of acoustic feature vectors.

Lesson 3 • HMM Training Algorithms

Covers the Baum-Welch and Viterbi algorithms for training and decoding. Connects algorithmic steps to practical parameter estimation.

Lesson 4 • Hidden Markov Models Explained

Defines HMM states, transitions, and observation probabilities. Students learn to model phoneme sequences as Markov chains with hidden states.

Lesson 1 • Convolutional Networks in Speech

Applies CNNs to spectrogram features for local pattern extraction. Shows how convolution reduces spectral variation before sequence modeling.

Lesson 2 • Connectionist Temporal Classification

Explains CTC loss for aligning variable-length outputs without explicit segmentation. Enables end-to-end training directly from audio to text.

Lesson 3 • Attention and Transformer Models

Introduces self-attention, multi-head attention, and the Transformer encoder-decoder. Positions Transformers as the current state-of-the-art backbone.

Lesson 4 • Recurrent Networks for Sequences

Introduces RNNs, LSTMs, and GRUs for modeling temporal speech sequences. Explains vanishing gradients and how gated units solve them.

Lesson 1 • Training Pipelines and Toolkits

Surveys open-source toolkits and describes training workflow configuration. Students gain hands-on familiarity with industry-standard tools.

Lesson 2 • System Architecture Overview

Maps all components of a modern recognition system and their interactions. Provides a blueprint students use when assembling complete pipelines.

Lesson 3 • Data Collection and Preparation

Covers corpus design, recording protocols, and transcription standards. Quality data preparation directly determines model performance.

Lesson 4 • Error Analysis and Debugging

Teaches systematic diagnosis of substitution, deletion, and insertion errors. Guides iterative improvement of recognition accuracy.

Lesson 1 • Noise Robustness Strategies

Applies multi-condition training, data augmentation, and feature normalization. Builds models that generalize across unseen noise environments.

Lesson 2 • Speaker Adaptation Methods

Covers MLLR, MAP, and i-vector adaptation for personalizing models. Reduces error rates when limited speaker-specific data is available.

Lesson 3 • Sources of Variability in Speech

Catalogs speaker, channel, and environmental variability that degrades accuracy. Motivates the adaptation techniques introduced in subsequent sections.

Lesson 4 • Transfer Learning and Fine-Tuning

Leverages pre-trained models and fine-tunes them on target domains. Dramatically reduces data and compute requirements for new applications.

Lesson 1 • Multilingual and Cross-Lingual Models

Explores multilingual pre-training and zero-shot cross-lingual transfer. Prepares students to build recognition systems for low-resource languages.

Lesson 2 • Whisper and Multitask Models

Examines Whisper's weakly supervised training on large-scale web data. Covers multitask conditioning for transcription, translation, and language identification.

Lesson 3 • Self-Supervised Learning Concepts

Explains contrastive learning, masked prediction, and representation learning without labels. Establishes why self-supervision is transformative for low-resource speech.

Lesson 4 • Fine-Tuning Pre-Trained Models

Provides a practical workflow for fine-tuning large models on custom datasets. Covers learning rate scheduling, gradient accumulation, and evaluation.

Lesson 1 • Streaming and Real-Time Recognition

Adapts batch models to streaming inference with chunk-based processing. Addresses latency, lookahead, and partial hypothesis management.

Lesson 2 • Model Optimization for Inference

Applies quantization, pruning, and knowledge distillation to reduce model size. Balances accuracy and latency for real-time deployment constraints.

Lesson 3 • Serving Infrastructure and APIs

Covers containerization, model serving frameworks, and REST or gRPC API design. Enables scalable, maintainable deployment of recognition services.

Lesson 4 • Edge and On-Device Deployment

Deploys compact models on mobile and embedded devices with limited resources. Covers hardware-aware optimization and on-device privacy benefits.