Introduction to Deep Learning

Module 1: Introduction to Deep Feedforward Networks

• • Gradient-based learning
  • Sigmoidal output units
  • Back propagation

Module 2: Regularization for Deep Learning

• • Regularization strategies
  • Noise injection
  • Ensemble methods
  • Dropout

Module 3: Optimization for Training Deep Models

• • Optimization algorithms: Gradient, Hessian-Free, Newton
  • Momentum
  • Batch normalization

Module 4: Convolutional Neural Networks

• • Convolutional kernels
  • Downsampled convolution
  • Zero padding
  • Backpropagating convolution

Module 5: Recurrent Neural Networks

• • Recurrence relationship & recurrent networks
  • Long short-term memory (LSTM)
  • Back propagation through time (BPTT)
  • Gated and simple recurrent units
  • Neural Turing machine (NTM)