Build a tiny character-level tokenizer with vocab, stoi/itos, and encode/decode helpers.
Get fluent with NumPy arrays, indexing, broadcasting, reductions, and numerically stable softmax.
Load the corpus, build batched (X, Y) sequences, and train a counting-based bigram model as a baseline.
Replace the count table with a learned weight matrix and derive cross-entropy, gradients, and SGD updates.
Implement forward and backward passes for linear, bias, ReLU, softmax+CE, and LayerNorm building blocks.
Add token and positional embeddings, then build masked single-head and multi-head self-attention with full backward passes.
Compose feed-forward networks, residual connections, and pre-LN Transformer blocks into the complete GPT forward/backward.
Implement Adam, wire up the full training and validation loop, then sample text with temperature and top-k decoding.