Deep Learning Bootcamp

“All knowledge is connected to all other knowledge. The fun is in making the connections.” — Arthur Aufderheide

The objective for the Deep Learning bootcamp is to ensure that the participants have enough theory and practical concepts of building a deep learning solution in the space of computer vision and natural language processing. Post the bootcamp, all the participants would be familiar with the following key concepts and would be able to apply them to a problem.

Key Concepts

• Theory: DL Formulation, Back-propagation, Activation, Optimization
• Paradigms: Supervised (Regression & Classification)
• Data: Images, Text (Sequences)
• Models: Sequential Models, Pre-trained Models (Transfer Learning, Word Embeddings, Language Models)
• Methods: Perceptron, Dense, Convolution - 1D & 2D, Recurrent - RNN & LSTM, Transformer, Embeddings, Data Augmentation, Regularisation
• Process: Setup, Encode, Design, Train, Select, Serve
• Tools: python-data-stack, keras, tensorflow

Approach

This would be a three-day instructor-led hands-on workshop to learn and implement an end-to-end deep learning model for computer vision (image recognition) and natural language processing (text classfication). This is predominantly a hands-on course and will be 70% programming/coding and 30% theory. There will be twelve sessions of two hours each over three days.

Session 1: Deep Learning (DL) Basics

• What is deep learning? Usage in Images (Computer Vision) & Text (NLP).
• DL for Supervised, Unsupervised & Reinforcement Learning
• Basic building blocks: Neurons, Activation functions, Back propagation, Loss, Optimisers
• Universal Approximation, MLP for tabular data

Session 2: DL for Images

• Introduction to problem and data-set
• Working on the cloud, including keras and tensorflow
• Data types, Loss types, Optimisers types
• Build a Multi-layer Perceptron (MLP) model

Session 3: Convolutional Neural Networks (CNN)

• Concept of Convolution & Max-pooling
• Build a CNN model for image classification
• Tricks to improve your model
  • Data Augmentation
  • Batch normalization / Dropouts

Session 4: Transfer Learning

• Concept of Transfer Learning
• Model Zoo and applicability of pre-trained models
• Leverage pre-trained model for image classification
• Deploying your DL model on the cloud

Session 5: Tips for DL Learning

• Dos & Don’t of good design of DL architecture: initialisation, scaling, filters & optimizers, learning rates
• Managing the learning process: Overfitting, learning rate change, batch changes, normalisation, etc.
• Using tensorboard visualisation to manage the process

Session 6: Annotation & Understanding DL Models

• Why understand & interpreting Deep Learning Models?
• Annotation: Generating Labelled Data
• Feature Visualisation by Optimization
• Spatial Attribution and Saliency Maps

Session 7: DL for Natural Language Processing

• Challenges with traditional NLP techniques
• Concept of Sparse vs. Dense Embedding
• Pre-trained Embedding: word2vec, glove, fastest
• Build MLP model using word embedding (for text classification)

Session 8: Recurrent Neural Networks (RNN)

• Concept of RNNs and Sequence Learning
• Concept of Long Short-Term Memory (LSTM)
• Build an LSTM model for text classification
• Limitation of LSTM models & LSTMVis

Session 9: CNN & Attention for Text

• Convolution on text: 1D variant
• Concept of attention on text
• Building an Attention Model for text classification
• Attention Visualisation

Session 10: Language Models for Text

• Transfer Learning for Text
• Intuition on Language Models
• Language Models: ELMo, ULMFit, Transformers
• Building a Language Model for text classification

Session 11: Text & Images Together

• Functional Model for Deep Learning
• Combining Multiple DL Models
• Generating Image and Text Annotation Together
• Serving DL Models

Session 12: Usage & Challenges in DL

• Images: GAN, Object Detection, Segmentation
• Text: Generation, Entity Detection, Co-Reference
• Unsupervised Learning: Autoencoders
• Challenges in building DL apps
• Way Forward & Learning Path

Resources

The material for the workshop is hosted on github: https://github.com/amitkaps/deep-learning

This is from the popular workshop series by the speakers on deep learning. Additional materials relevant to learning Deep Learning would be shared prior to the workshop.

Target Audience

• A machine learning practitioner
• A programmer interested in building data science products
• Anyone (researcher, student, professional) learning deep learning
• Corporates and start-ups looking to add DL to their product or service offerings

Pre-requisites

• This is a hands-on course and hence, participants should be comfortable with programming. Familiarity with python data stack is ideal.
• Prior knowledge of machine learning will be helpful. Participants should have some practice with basic machine learning problems e.g. regression, classification.
• While the deep learning concepts will be taught in an intuitive way, some prior knowledge of linear algebra and calculus would be helpful.

Software Requirements

We will be using Python data stack for the workshop with keras and tensorflow for the deep learning component. Please install Ananconda for Python 3 for the workshop. Additional requirement will be communicated to participants.

Facilitators’ Profile

Amit Kapoor teaches the craft of telling visual stories with data. He conducts workshops and trainings on Data Science in Python and R, as well as on Data Visualisation topics. His background is in strategy consulting having worked with AT Kearney in India, then with Booz & Company in Europe and more recently for startups in Bangalore. He did his B.Tech in Mechanical Engineering from IIT, Delhi and PGDM (MBA) from IIM, Ahmedabad. You can find more about him at http://amitkaps.com/ and tweet him at @amitkaps.

Bargava Subramanian is a practicing Data Scientist. He has 14 years of experience delivering business analytics solutions to Investment Banks, Entertainment Studios and High-Tech companies. He has given talks and conducted workshops on Data Science, Machine Learning, Deep Learning and Optimization in Python and R. He has a Masters in Statistics from University of Maryland, College Park, USA. He is an ardent NBA fan. You can tweet to him at @bargava.