Learn Deep Learning with 66 Flashcards: From the Perceptron to Diffusion Models

Deck EV4453 — Deep Learning packs 66 cards cut from the encyclopedic overview of the field: architectures, training tricks, hardware and a surprising amount of history. It is a solid map of the vocabulary, but the history cards are scattered from card 11 to card 66, and a few core ideas are used without ever being defined. Here is how to turn it into real understanding.

What the cards actually say

Everything the deck covers

Architectures: CNN, RNN, LSTM (with its 1999 forget gate), Transformer, GAN, StyleGAN, ResNet, Highway Network, VGG-16, Inception v3, LeNet, DanNet, AlexNet, Deep Belief Network, Restricted and Deep Boltzmann Machines, feedforward network, perceptron, NeRF. Training: backpropagation, stochastic gradient descent, learning rate, batch learning, ReLU, sigmoid, dropout, weight decay, data augmentation, regularization, overfitting, transfer learning, fine-tuning, representation learning, CTC, predictive coding. Context: GPUs, TPUs, NPUs, ImageNet, MNIST, TIMIT, the universal approximation theorem, and applications from computer vision and NLP to machine translation and neural style transfer.

Tip 1: rebuild the timeline yourself

The deck mentions a dozen dates but never in order: the Ising model sits at card 61 and the 1958 perceptron at card 62, after Stable Diffusion. Make one extra card per era and review them as a chain: 1920s Ising model → 1958 perceptron → 1969 ReLU (Fukushima) → 1989 LeNet → 1997 LSTM / 1999 forget gate → 2006 Deep Belief Nets and CTC → 2011 DanNet → 2012 AlexNet → 2015 ResNet, Highway Nets, diffusion → 2018 StyleGAN → 2022 DALL-E 2 and Stable Diffusion. Two facts the deck leaves out complete it: the Transformer dates from 2017 ("Attention Is All You Need"), and LSTM from 1997 (Hochreiter and Schmidhuber).

Fix one card while you do it: card 45 dates the Deep Boltzmann Machine to "the 1980s-1990s". Boltzmann machines are from the 1980s, but the deep variant was introduced by Salakhutdinov and Hinton in 2009, after the 2006 Deep Belief Network.

Tip 2: write the missing foundation cards

Several backs lean on terms no front defines. Add these before card 1: loss function, gradient descent (card 63 only gives the stochastic version), attention (the Transformer card depends on it), softmax, epoch, and batch normalization. Card 5 also deserves a rewrite: "data can flow in any direction" is vague; an RNN feeds its hidden state back into itself at the next time step.

Finally, group the four regularization cards (dropout, weight decay, data augmentation, regularization) and quiz them as "which technique does X?" so they stop blurring together.

Keep going

Pair this deck with the classic ML metrics covered in Remember Every Machine Learning Metric, then search EV4453 in the Flip It library to start reviewing.