Deep Learning Bootcamp
“All knowledge is connected to all other knowledge. The fun is in making the connections.” — Arthur Aufderheide
The objective for the Deep Learning bootcamp is to ensure that the participants have enough theory and practical concepts of building a deep learning solution in the space of computer vision and natural language processing. Post the bootcamp, all the participants would be familiar with the following key concepts and would be able to apply them to a problem.
- Theory: DL Formulation, Back-propagation, Activation, Optimization
- Paradigms: Supervised (Regression & Classification)
- Data: Images, Text (Sequences)
- Models: Sequential Models, Pre-trained Models (Transfer Learning, Word Embeddings, Language Models)
- Methods: Perceptron, Dense, Convolution - 1D & 2D, Recurrent - RNN & LSTM, Transformer, Embeddings, Data Augmentation, Regularisation
- Process: Setup, Encode, Design, Train, Select, Serve
This would be a three-day instructor-led hands-on workshop to learn and implement an end-to-end deep learning model for computer vision (image recognition) and natural language processing (text classfication). This is predominantly a hands-on course and will be 70% programming/coding and 30% theory. There will be twelve sessions of two hours each over three days.
Session 1: Deep Learning (DL) Basics
- What is deep learning? Usage in Images (Computer Vision) & Text (NLP).
- DL for Supervised, Unsupervised & Reinforcement Learning
- Basic building blocks: Neurons, Activation functions, Back propagation, Loss, Optimisers
- Universal Approximation, MLP for tabular data
Session 2: DL for Images
- Introduction to problem and data-set
- Working on the cloud, including keras and tensorflow
- Data types, Loss types, Optimisers types
- Build a Multi-layer Perceptron (MLP) model
Session 3: Convolutional Neural Networks (CNN)
- Concept of Convolution & Max-pooling
- Build a CNN model for image classification
- Tricks to improve your model
- Data Augmentation
- Batch normalization / Dropouts
Session 4: Transfer Learning
- Concept of Transfer Learning
- Model Zoo and applicability of pre-trained models
- Leverage pre-trained model for image classification
- Deploying your DL model on the cloud
Session 5: Tips for DL Learning
- Dos & Don’t of good design of DL architecture: initialisation, scaling, filters & optimizers, learning rates
- Managing the learning process: Overfitting, learning rate change, batch changes, normalisation, etc.
- Using tensorboard visualisation to manage the process
Session 6: Annotation & Understanding DL Models
- Why understand & interpreting Deep Learning Models?
- Annotation: Generating Labelled Data
- Feature Visualisation by Optimization
- Spatial Attribution and Saliency Maps
Session 7: DL for Natural Language Processing
- Challenges with traditional NLP techniques
- Concept of Sparse vs. Dense Embedding
- Pre-trained Embedding: word2vec, glove, fastest
- Build MLP model using word embedding (for text classification)
Session 8: Recurrent Neural Networks (RNN)
- Concept of RNNs and Sequence Learning
- Concept of Long Short-Term Memory (LSTM)
- Build an LSTM model for text classification
- Limitation of LSTM models & LSTMVis
Session 9: CNN & Attention for Text
- Convolution on text: 1D variant
- Concept of attention on text
- Building an Attention Model for text classification
- Attention Visualisation
Session 10: Language Models for Text
- Transfer Learning for Text
- Intuition on Language Models
- Language Models: ELMo, ULMFit, Transformers
- Building a Language Model for text classification
Session 11: Text & Images Together
- Functional Model for Deep Learning
- Combining Multiple DL Models
- Generating Image and Text Annotation Together
- Serving DL Models
Session 12: Usage & Challenges in DL
- Images: GAN, Object Detection, Segmentation
- Text: Generation, Entity Detection, Co-Reference
- Unsupervised Learning: Autoencoders
- Challenges in building DL apps
- Way Forward & Learning Path
The material for the workshop is hosted on github: https://github.com/amitkaps/deep-learning
This is from the popular workshop series by the speakers on deep learning. Additional materials relevant to learning Deep Learning would be shared prior to the workshop.
- A machine learning practitioner
- A programmer interested in building data science products
- Anyone (researcher, student, professional) learning deep learning
- Corporates and start-ups looking to add DL to their product or service offerings
- This is a hands-on course and hence, participants should be comfortable with programming. Familiarity with python data stack is ideal.
- Prior knowledge of machine learning will be helpful. Participants should have some practice with basic machine learning problems e.g. regression, classification.
- While the deep learning concepts will be taught in an intuitive way, some prior knowledge of linear algebra and calculus would be helpful.
We will be using Python data stack for the workshop with
tensorflow for the deep learning component. Please install Ananconda for Python 3 for the workshop. Additional requirement will be communicated to participants.
Amit Kapoor teaches the craft of telling visual stories with data. He conducts workshops and trainings on Data Science in Python and R, as well as on Data Visualisation topics. His background is in strategy consulting having worked with AT Kearney in India, then with Booz & Company in Europe and more recently for startups in Bangalore. He did his B.Tech in Mechanical Engineering from IIT, Delhi and PGDM (MBA) from IIM, Ahmedabad. You can find more about him at http://amitkaps.com/ and tweet him at @amitkaps.
Bargava Subramanian is a practicing Data Scientist. He has 14 years of experience delivering business analytics solutions to Investment Banks, Entertainment Studios and High-Tech companies. He has given talks and conducted workshops on Data Science, Machine Learning, Deep Learning and Optimization in Python and R. He has a Masters in Statistics from University of Maryland, College Park, USA. He is an ardent NBA fan. You can tweet to him at @bargava.