Deep Learning Tutorial

45 Papers + TF implementations

Topics (papers)

Modern CNNs

• Alex Krizhevsky, et al. "ImageNet Classification with Deep Convolutional Neural Networks", NIPS, 2012
• Christian Szegedy, et al. "Going Deeper with Convolutions", CVPR, 2015
• Christian Szegedy, et al. "Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning", ArXiv, 2016
• Kaiming He, et al. "Deep Residual Learning for Image Recognition", CVPR, 2016
• Andreas Veit, et al. "Residual Networks are Exponential Ensembles of Relatively Shallow Networks", ArXiv, 2016
• Sergey Zagoruyko and Nikos Komodakis "Wide Residual Networks", ArXiv, 2016

Regularization

• Nitish Srivastava, et al. "Dropout- A Simple Way to Prevent Neural Networks from Overfitting", JMLR, 2014
• Sergey Ioffe and Christian Szegedy "Batch Normalization- Accelerating Deep Network Training by Reducing Internal Covariate Shift, ArXiv, 2015

Algorithms behind AlphaGo

• David Silver et al. "Mastering the game of Go with deep neural networks and tree search", Nature, 2016

Optimization Methods

• Momentum, NAG, AdaGrad, AdaDelta, RMSprop, ADAM
• Diederik Kingma and Jimmy Bam "ADAM: A Method For Stochastic Optimization", ICLR, 2015

Restricted Boltzmann Machine

• Geoffrey Hinton, "A Practical Guide to Training Restricted Boltzmann Machines", 2010

Semantic Segmentation

• Jonathan Long et al. "Fully Convolutional Networks for Semantic Segmentation", CVPR, 2015
• Liang-Chieh Chen et al. "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", CVPR, 2015
• Hyeonwoo Noh et al. "Learning Deconvolution Network for Semantic Segmentation", ICCV, 2015
• Liang-Chieh Chen et al. "DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs", ArXiv, 2016

Weakly Supervised Localization

• Maxime Oquab et al. "Is object localization for free? – Weakly-supervised learning with convolutional neural networks", CVPR, 2015
• Bolei Zhou et al. "Learning Deep Features for Discriminative Localization", CVPR, 2016

Image detection methods

• Ross Girshick et al. "Rich feature hierarchies for accurate object detection and semantic segmentation", CVPR, 2014
• Kaiming He et al. "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", CVPR, 2015
• Ross Girshick, "Fast R-CNN", ICCV, 2015
• Shaoqing Ren et al. "Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks", NIPS, 2015
• Joseph Redmon et al. "You Only Look Once: Unified, Real-Time Object Detection", CVPR, 2016
• Donggeun Yoo et al. "AttentionNet: Aggregating Weak Directions for Accurate Object Detection", ICCV, 2015
• Wei Liu et al. "SSD: Single Shot MultiBox Detector", ECCV, 2016
• Joseph Redmon, Ali Farhadi, "YOLO9000: Better, Faster, Stronger", ArXiv, 2017

Visual Q&A

• Hyeonwoo Noh et al. "Image Question Answering using Convolutional Neural Network with Dynamic Parameter Prediction", CVPR, 2015
• Akira Fukui et al. "Multimodal Compact Bilinear Pooling for VQA", CVPR, 2016

Deep reinforcement learning

• Volodymyr Mnih et al. "Playing Atari with Deep Reinforcement Learning", NIPS, 2013
• Hado van Hasselt et al. "Deep Reinforcement Learning with Double Q-learning", AAAI, 2016

Recurrent Neural Networks

• Alex Graves, "Generating Sequences With Recurrent Neural Networks", ArXiv, 2013

Word embedding

• Tomas Mikolov et al. "Distributed Representations of Words and Phrases and their Compositionality", NIPS, 2013

Image captioning

• Oriol Vinyals et al. "Show and Tell: A Neural Image Caption Generator", CVPR, 2015
• Kelvin Xu et al. "Show, Attend and Tell: Neural Image Caption Generation with Visual Attention", ICML, 2015
• Justin Johnson et al. "DenseCap: Fully Convolutional Localization Networks for Dense Captioning", CVPR, 2016

Neural Styles

• Leon A. Gatys et al. "Texture Synthesis Using Convolutional Neural Networks", NIPS, 2015
• Aravindh Mahendran and Andrea Vedaldi, "Understanding Deep Image Representations by Inverting Them", CVPR, 2015
• Leon A. Gatys et al. "A Neural Algorithm of Artistic Style", ArXiv, 2015

Generative adversarial networks

• Ian J. Goodfellow et al. "Generative Adversarial Networks", NIPS, 2015
• Alec Radford et al. "Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks", ICLR, 2016
• Scott Reed et al. "Generative Adversarial Text to Image Synthesis", ICML, 2016
• Donggeun Yoo et al. "Pixel Level Domain Transfer", ECCV, 2016
• Phillip Isola et al, "Image-to-Image Translation with Conditional Adversarial Networks", ArXiv, 2016
• Anh Nguyen et al. "Plug & Play Generative Networks: Conditional Iterative Generation of Images in Latent Space", ArXiv, 2016
• Scott Reed et al. "Learning What and Where to Draw", NIPS, 2016

and implementations (which can be found in TF-101)

• Basic Python usage (numpy, matplotlib, ..)
• Handling MNIST
• Logistic regression
• Multilayer Perceptron
• Convolutional Neural Network
• Denoising Autoencoders (+Convolutional)
• Class Activation Map
• Semantic Segmentation
• Using Custom Dataset
• Recurrent Neural Network
• Char-RNN
• Word2Vec
• Neural Style