Generative AI Models and GPU Systems Course

Lesson 1 • Neural Network Architecture Basics

Introduces neurons, layers, and activation functions. Connects perceptron theory to modern multi-layer networks.

Lesson 2 • Evaluating Model Performance

Covers standard metrics for classification and regression tasks. Enables students to interpret results and diagnose model weaknesses.

Lesson 3 • Core Concepts in Supervised Learning

Covers labeled datasets, loss functions, and gradient descent. Provides the mathematical intuition underlying all neural network training.

Lesson 4 • AI, ML, and Deep Learning Hierarchy

Defines the nested relationship among AI, ML, and deep learning. Establishes vocabulary used throughout the course.

Lesson 1 • Selecting GPUs for AI Workloads

Provides a framework for matching GPU specs to model size and training budget. Students can justify hardware procurement decisions.

Lesson 2 • GPU Interconnects and Multi-GPU Systems

Covers high-bandwidth interconnects and peer-to-peer GPU communication. Prepares students to design multi-GPU training clusters.

Lesson 3 • CPU vs. GPU Computational Models

Contrasts serial CPU execution with massively parallel GPU execution. Establishes why GPUs accelerate matrix-heavy AI workloads.

Lesson 4 • GPU Internal Architecture

Examines streaming multiprocessors, CUDA cores, and tensor cores. Connects hardware units to specific deep learning operations.

Lesson 1 • Writing Efficient CUDA Kernels

Focuses on warp divergence, occupancy, and shared memory usage. Students learn to write kernels that maximize GPU utilization.

Lesson 2 • CUDA Programming Model Essentials

Introduces the host-device model, kernels, and thread hierarchy. Provides the conceptual framework for all subsequent CUDA coding.

Lesson 3 • Profiling and Debugging GPU Code

Introduces profiling tools and common GPU debugging techniques. Enables students to identify and resolve performance bottlenecks.

Lesson 4 • CUDA Libraries for Deep Learning

Surveys cuBLAS, cuDNN, and cuFFT for accelerated AI primitives. Connects library calls to neural network layer implementations.

Lesson 1 • Distributed Training with Frameworks

Introduces data-parallel and model-parallel strategies within frameworks. Prepares students for large-scale multi-GPU training runs.

Lesson 2 • Framework Overview and GPU Backend

Compares leading frameworks and their GPU execution backends. Students understand how frameworks abstract CUDA for model development.

Lesson 3 • Tensor Operations and Autograd

Covers tensor creation, device placement, and automatic differentiation. Connects tensor math to gradient computation in training loops.

Lesson 4 • Building Models with Framework APIs

Teaches layer stacking, custom modules, and model serialization. Students construct reusable model components for generative architectures.

Lesson 1 • Diffusion Models and Score Matching

Introduces forward noising, reverse denoising, and score-based generation. Students grasp why diffusion models achieve high-fidelity outputs.

Lesson 2 • Generative vs. Discriminative Models

Distinguishes generative from discriminative modeling objectives. Sets the conceptual foundation for all generative architectures covered ahead.

Lesson 3 • Generative Adversarial Networks

Covers generator-discriminator dynamics, loss functions, and training instability. Students understand GAN variants and their practical limitations.

Lesson 4 • Variational Autoencoders

Explains encoder-decoder structure, reparameterization trick, and ELBO loss. Connects probabilistic latent spaces to controlled generation.

Lesson 1 • Mixed Precision and Quantization in Training

Applies FP16, BF16, and INT8 formats to reduce memory and increase throughput. Students implement mixed-precision training with loss scaling.

Lesson 2 • Scaling Laws and Model Size Planning

Covers empirical scaling laws relating compute, data, and model size. Students use these laws to plan training budgets before execution.

Lesson 3 • Advanced Parallelism Strategies

Extends data parallelism to tensor and pipeline parallelism for massive models. Students configure multi-dimensional parallelism for transformer training.

Lesson 4 • Memory Optimization Techniques

Teaches gradient checkpointing, activation recomputation, and offloading. Enables training of models that exceed single-GPU VRAM capacity.

Lesson 1 • Transfer Learning for Generative AI

Explains pretrained model reuse, feature extraction, and domain shift. Motivates fine-tuning as a compute-efficient alternative to training from scratch.

Lesson 2 • Domain-Specific Fine-Tuning Pipelines

Builds end-to-end pipelines for text, image, and multimodal domain adaptation. Students apply fine-tuning to real-world domain datasets.

Lesson 3 • Preventing Catastrophic Forgetting

Covers elastic weight consolidation, replay buffers, and continual learning. Students preserve pretrained knowledge while acquiring new task capabilities.

Lesson 4 • Parameter-Efficient Fine-Tuning Methods

Covers LoRA, prefix tuning, and adapter layers for low-cost adaptation. Students implement PEFT methods that update a fraction of model parameters.

Lesson 1 • Cloud and On-Premises GPU Deployment

Compares cloud GPU instances with on-premises clusters for cost and control. Students select and configure deployment environments for production workloads.

Lesson 2 • Inference Serving Frameworks

Surveys GPU-optimized serving frameworks and batching strategies. Students configure serving stacks for high-concurrency generative workloads.

Lesson 3 • Containerization and Orchestration

Packages GPU workloads in containers and deploys them with orchestration tools. Students automate scaling and rolling updates for inference services.

Lesson 4 • Model Optimization for Inference

Applies quantization, pruning, and graph optimization to reduce inference cost. Students produce deployment-ready models with minimal accuracy loss.