🧠

Deep Learning

66 cards · 🇺🇸 English

Neural network architectures, training concepts, and deep learning terminology.

Study all 66 cards for free

Flashcards, quizzes, pixel-art games and a daily review that remembers for you.

Study this deck free

Try the first 10 cards

1 / 10
Question

Deep learning

Knew itLearning

Tap the card for the answer · swipe right if you knew it, left if not

Already have an account? Sign in

Cards

  1. 1Deep learning
    A subset of machine learning that uses multilayered neural networks (typically 3+ layers) to perform tasks like classification, regression, and representation learning
  2. 2Why is it called "deep" learning?
    The "deep" refers to the use of multiple layers (ranging from three to several hundred or thousands) in the neural network
  3. 3Three main learning paradigms in deep learning
    Supervised learning, semi-supervised learning, and unsupervised learning
  4. 4Convolutional Neural Network (CNN)
    A deep learning architecture particularly effective for computer vision tasks, using convolutional layers to process spatial data like images
  5. 5Recurrent Neural Network (RNN)
    A neural network architecture where data can flow in any direction, commonly used for sequential data like language modeling and time series
  6. 6LSTM
    Long Short-Term Memory, an RNN architecture designed to learn long-term dependencies and handle the vanishing gradient problem
  7. 7Transformer
    A deep learning architecture that relies on attention mechanisms, widely used in natural language processing (e.g., GPT, BERT)
  8. 8GAN
    Generative Adversarial Network - two neural networks that contest with each other: a generator creates data and a discriminator evaluates it
  9. 9Backpropagation
    An efficient application of the chain rule for computing gradients, used to train neural networks by propagating errors backward through layers
  10. 10Vanishing gradient problem
    A difficulty in training deep networks where gradients become extremely small in early layers, preventing effective learning
  11. 11ReLU activation function
    Rectified Linear Unit - an activation function that outputs the input directly if positive, otherwise zero. Introduced by Kunihiko Fukushima in 1969
  12. 12Deep Belief Network (DBN)
    A generative model composed of stacked restricted Boltzmann machines, developed by Geoffrey Hinton and colleagues in 2006
  13. 13Dropout in deep learning
    A regularization technique that randomly omits units from hidden layers during training to prevent overfitting
  14. 14Credit Assignment Path (CAP)
    The chain of transformations from input to output in a neural network, describing potentially causal connections
  15. 15What CAP depth is generally considered 'deep' learning?
    Most researchers agree that deep learning involves CAP depth higher than two
  16. 16Feature engineering
    The manual process of transforming raw data into suitable representations for machine learning algorithms
  17. 17How does deep learning differ from traditional machine learning in feature handling?
    Deep learning automatically discovers useful feature representations from data, eliminating the need for hand-crafted features
  18. 18Universal Approximation Theorem
    A theorem stating that feedforward neural networks with a single hidden layer can approximate any continuous function
  19. 19LeNet
    A CNN created by Yann LeCun in 1989 for recognizing handwritten ZIP codes on mail
  20. 20What year did LeNet-5 get applied to check recognition by banks?
    1998 - it classified handwritten digits in 32x32 pixel images
  21. 21AlexNet
    A CNN by Krizhevsky, Sutskever, and Hinton that won ImageNet 2012 by a significant margin, marking the start of the deep learning revolution
  22. 22What hardware advancement enabled the deep learning revolution?
    GPUs (Graphics Processing Units), which dramatically accelerated neural network training
  23. 23ImageNet
    A large-scale visual recognition competition and dataset that drove advances in computer vision and deep learning
  24. 24ResNet
    Residual Neural Network (2015) - uses residual connections to train very deep networks (100+ layers) by addressing the degradation problem
  25. 25Highway Network
    An architecture published in May 2015 that enables training of very deep networks using gating mechanisms
  26. 26Neural style transfer
    A technique (2015) that applies the artistic style of one image to the content of another using pretrained CNNs
  27. 27Google DeepDream
    A 2015 computer vision program that uses CNNs to generate artistic, dream-like images by enhancing patterns
  28. 28StyleGAN
    A GAN architecture by Nvidia (2018) that achieves excellent image quality in generative modeling
  29. 29Diffusion Models
    Generative models (2015) that gradually add and remove noise, eclipsing GANs since ~2022 (used in DALL-E 2, Stable Diffusion)
  30. 30DALL-E 2
    An AI system by OpenAI (2022) that generates images from text descriptions using diffusion models

And 36 more cards. Study the whole deck free

Articles about this deck

More Programming decks

See all
Study all 66 cards free