AI Infrastructure: Introduction to the AI Hypercomputer Course

Lesson 1 • What Is AI Infrastructure

Defines AI infrastructure and distinguishes it from traditional IT stacks. Anchors the chapter by framing every subsequent topic within a unified system view.

Lesson 2 • Core Components of AI Systems

Maps the hardware, software, and networking layers that compose an AI system. Provides the component taxonomy used throughout the course.

Lesson 3 • Introduction to the AI Hypercomputer Concept

Introduces the AI Hypercomputer as a tightly integrated, purpose-built system for large-scale AI. Sets the architectural vision that the course builds toward.

Lesson 1 • Accelerator Performance Benchmarking

Introduces standard benchmarks and profiling tools for AI accelerators. Enables students to interpret vendor specifications critically.

Lesson 2 • TPUs and Domain-Specific Accelerators

Covers tensor processing units and other purpose-built chips optimised for matrix operations. Contrasts their trade-offs against general-purpose GPUs.

Lesson 3 • GPU Architecture for AI

Explains GPU microarchitecture elements critical to AI throughput. Grounds hardware selection decisions in architectural understanding.

Lesson 4 • Multi-Accelerator Node Design

Examines how multiple accelerators are packaged within a single server node. Connects node-level design to cluster-level performance outcomes.

Lesson 1 • Networking Requirements of Distributed AI

Quantifies bandwidth and latency demands imposed by distributed training algorithms. Establishes why standard data-centre networking is insufficient for AI.

Lesson 2 • Network Topology Design

Covers fat-tree, dragonfly, and torus topologies used in large AI clusters. Teaches students to select topology based on traffic patterns and scale.

Lesson 3 • In-Network Computing

Introduces compute offload capabilities embedded in modern network switches. Shows how in-network reduction accelerates collective operations.

Lesson 4 • RDMA and High-Performance Fabrics

Explains Remote Direct Memory Access and its role in bypassing CPU overhead. Connects RDMA to the fabric technologies used in AI Hypercomputers.

Lesson 1 • Parallel and Distributed File Systems

Covers architectures of high-performance parallel file systems designed for HPC and AI. Connects file system design to sustained throughput at scale.

Lesson 2 • Storage Tiering and Caching

Introduces multi-tier storage hierarchies that balance cost and performance. Shows how caching layers reduce pressure on primary storage during training.

Lesson 3 • AI Storage Requirements and Patterns

Characterises the I/O patterns of dataset ingestion, checkpointing, and model serving. Frames storage design choices around AI-specific access behaviour.

Lesson 4 • Object Storage for AI Datasets

Explains object storage semantics and their fit for large, immutable AI datasets. Addresses integration patterns between object stores and training pipelines.

Lesson 1 • Debugging and Profiling Distributed Jobs

Provides systematic methods for diagnosing hangs, stragglers, and memory errors in distributed runs. Builds operational confidence for managing large training jobs.

Lesson 2 • Collective Communication Operations

Details all-reduce, all-gather, and broadcast operations used during distributed training. Connects communication primitives to gradient synchronisation workflows.

Lesson 3 • Distributed Training Fundamentals

Explains data parallelism, model parallelism, and pipeline parallelism from first principles. Provides the conceptual foundation for all framework-level decisions.

Lesson 4 • Training Framework Architecture

Surveys leading distributed training frameworks and their design philosophies. Enables students to select and configure frameworks for specific workloads.

Lesson 1 • Performance Modelling and Capacity Planning

Introduces analytical models for predicting system throughput and utilisation. Enables students to size Hypercomputer deployments for target workloads.

Lesson 2 • Pod and Cluster Organisation

Describes how accelerator pods, racks, and clusters are organised hierarchically. Connects physical layout to network topology and fault domain design.

Lesson 3 • Hypercomputer Architecture Principles

Defines the architectural pillars of an AI Hypercomputer: co-design, tight integration, and software-hardware co-optimisation. Frames the system-level view for the chapter.

Lesson 4 • Interconnect Fabric Integration

Details how high-speed fabrics are integrated across compute, storage, and management planes. Reinforces networking knowledge within the full system context.

Lesson 1 • Job Scheduling Strategies

Covers gang scheduling, preemption, and priority queues for AI training jobs. Connects scheduling policy to cluster utilisation and fairness outcomes.

Lesson 2 • Multi-Tenant Cluster Management

Addresses isolation, quota enforcement, and resource sharing across multiple teams. Ensures students can manage shared infrastructure without contention.

Lesson 3 • Orchestration Fundamentals for AI

Introduces container orchestration concepts adapted for GPU-intensive AI workloads. Establishes the scheduling vocabulary used throughout the chapter.

Lesson 4 • Cluster Observability and Alerting

Builds a monitoring stack for GPU utilisation, job health, and infrastructure metrics. Provides the operational visibility needed to manage production AI clusters.

Lesson 1 • Capacity and Change Management

Establishes processes for infrastructure changes, firmware updates, and capacity expansion. Ensures operational stability as the Hypercomputer scales over time.

Lesson 2 • Fault Tolerance and High Availability

Covers redundancy patterns, failure detection, and automatic recovery for AI infrastructure. Directly addresses the reliability requirements of long-running training jobs.

Lesson 3 • Data Security and Access Control

Addresses encryption, access policies, and audit logging for training datasets and model artefacts. Connects data governance requirements to infrastructure controls.

Lesson 4 • Security Architecture for AI Infrastructure

Defines the threat model and security controls specific to AI compute environments. Ensures students can protect model weights, data, and compute resources.