Natural Language Processing Course

Lesson 1 • Linguistic Essentials for NLP

Covers phonology, morphology, syntax, semantics, and pragmatics as they apply to computational text analysis. Provides the linguistic grounding needed to understand why text processing is non-trivial.

Lesson 2 • Text Representation: Bag-of-Words and TF-IDF

Introduces count-based and frequency-weighted vector representations of text. Students gain hands-on experience converting raw documents into numerical feature matrices.

Lesson 3 • Text Preprocessing Fundamentals

Teaches tokenization, normalization, stop-word removal, and stemming as essential data-cleaning steps. These techniques directly affect the quality of every downstream NLP model.

Lesson 4 • Evaluation Metrics for NLP Tasks

Presents precision, recall, F1-score, accuracy, and perplexity as standard measures for assessing NLP system performance. Connects metric selection to specific task types introduced later.

Lesson 1 • Naive Bayes Text Classification

Applies the Naive Bayes classifier to sentiment analysis and spam detection tasks. Connects probabilistic assumptions to practical classification pipelines built in the previous sections.

Lesson 2 • Probability Theory Review for NLP

Refreshes conditional probability, Bayes' theorem, and joint distributions in the context of language data. Provides the mathematical foundation for all probabilistic models in this chapter.

Lesson 3 • Smoothing and Backoff Techniques

Addresses the zero-probability problem in sparse language data using Laplace, Kneser-Ney, and backoff strategies. Demonstrates how smoothing improves generalization to unseen word sequences.

Lesson 4 • Hidden Markov Models for Sequence Labeling

Introduces HMMs for part-of-speech tagging and named entity recognition as sequence labeling problems. Students implement the Viterbi algorithm to decode the most probable label sequence.

Lesson 1 • GloVe and FastText Embeddings

Presents GloVe's global co-occurrence matrix factorization and FastText's subword-level representations. Compares their strengths for handling rare words and morphologically rich languages.

Lesson 2 • Using Pretrained Embeddings in NLP Pipelines

Demonstrates how to load, fine-tune, and integrate pretrained embeddings into text classification and sequence labeling models. Bridges static embeddings to the contextual models introduced in later chapters.

Lesson 3 • Evaluating and Analyzing Word Embeddings

Applies intrinsic benchmarks such as word analogy and similarity tasks alongside extrinsic downstream evaluation. Addresses bias and fairness concerns embedded in learned vector spaces.

Lesson 4 • From Sparse to Dense Representations

Contrasts one-hot encoding and TF-IDF vectors with dense embeddings to motivate the shift in representation strategy. Explains the distributional hypothesis as the theoretical basis for word vectors.

Lesson 1 • Recurrent Neural Networks and LSTMs

Builds RNNs and LSTMs for sequential text modeling, addressing the vanishing gradient problem. Students implement sequence classification and language modeling with recurrent architectures.

Lesson 2 • Neural Network Basics for Text

Reviews feedforward networks, activation functions, backpropagation, and gradient descent as applied to NLP input formats. Establishes the deep learning vocabulary used throughout this chapter.

Lesson 3 • Attention Mechanisms in Neural NLP

Explains additive and multiplicative attention as solutions to the information bottleneck in seq2seq models. Prepares students for the self-attention mechanism central to Transformer architectures.

Lesson 4 • Sequence-to-Sequence Models

Introduces the encoder-decoder framework for machine translation, summarization, and dialogue generation. Students implement a basic seq2seq model with teacher forcing and beam search decoding.

Lesson 1 • Fine-Tuning Pretrained Models

Demonstrates task-specific fine-tuning of pretrained Transformers for classification, NER, and question answering. Covers learning rate scheduling, gradient clipping, and early stopping for stable fine-tuning.

Lesson 2 • The Transformer Architecture in Depth

Dissects multi-head self-attention, positional encoding, feed-forward sublayers, and layer normalization. Students trace a token through the full encoder and decoder stacks to build mechanistic understanding.

Lesson 3 • Tokenization in Transformer Models

Explains byte-pair encoding, WordPiece, and SentencePiece tokenizers used in modern pretrained models. Addresses subword vocabulary construction and its effect on multilingual and domain-specific tasks.

Lesson 4 • Pretraining Objectives and Strategies

Covers masked language modeling, causal language modeling, and next-sentence prediction as self-supervised pretraining objectives. Explains how large-scale pretraining creates transferable language representations.

Lesson 1 • Coreference Resolution

Identifies mentions that refer to the same real-world entity across a document using span-based models. Explains how coreference chains improve downstream tasks such as summarization and QA.

Lesson 2 • Information Extraction Pipelines

Integrates NER, relation extraction, and coreference into a unified information extraction system. Students build a pipeline that populates a structured knowledge store from unstructured text.

Lesson 3 • Text Classification and Sentiment Analysis

Covers binary and multi-class text classification with both traditional and Transformer-based models. Applies aspect-level sentiment analysis to product reviews and social media data.

Lesson 4 • Named Entity Recognition

Trains sequence labeling models to identify persons, organizations, locations, and domain-specific entities. Covers BIO tagging schemes and evaluation with span-level F1 metrics.

Lesson 1 • Open-Domain Conversational AI

Explores retrieval-based and generative approaches to open-domain chatbots, including persona conditioning and safety filtering. Evaluates conversational quality with automatic and human metrics.

Lesson 2 • Decoding Strategies for Text Generation

Compares greedy, beam search, top-k, nucleus, and temperature-based sampling for controlling generation diversity. Students tune decoding parameters to balance fluency, diversity, and factual accuracy.

Lesson 3 • Abstractive Text Summarization

Applies encoder-decoder Transformers to generate concise, fluent summaries of long documents. Addresses faithfulness, hallucination, and factual consistency as key quality dimensions.

Lesson 4 • Task-Oriented Dialogue Systems

Designs dialogue state tracking, policy learning, and natural language generation components for goal-directed conversations. Covers slot filling, belief state representation, and end-to-end trainable dialogue models.

Lesson 1 • Responsible Deployment of LLMs

Addresses hallucination, bias, privacy leakage, and adversarial prompt injection as deployment risks. Students apply mitigation strategies and design evaluation frameworks for production LLM systems.

Lesson 2 • Scaling Laws and Emergent Capabilities

Examines how model size, data volume, and compute interact to produce emergent NLP capabilities. Connects scaling law research to practical decisions about model selection and resource allocation.

Lesson 3 • Prompt Engineering and In-Context Learning

Covers zero-shot, few-shot, chain-of-thought, and instruction prompting techniques for eliciting desired LLM behavior. Students design and evaluate prompt templates for classification, extraction, and generation tasks.

Lesson 4 • Reinforcement Learning from Human Feedback

Explains the RLHF pipeline: supervised fine-tuning, reward model training, and PPO-based policy optimization. Connects alignment objectives to measurable improvements in helpfulness and harmlessness.