Syllabus (COMPUTER APPLICATION)

Course Type: ME-7

Semester: 7

Course Code: BBCAMEA47C

Course Title: Deep Learning

(L-P-Tu): 3-1-0

Credit: 4

Practical/Theory: Combined

Course Objective: Course Objectives: The main objective of this course is to: • Provide the foundations of the practical implementation and usage of deep learning. • Evaluate, in the context of a case study, the advantages and disadvantages of Neural Networks architectures and other approaches. • Implement deep learning models in Python using the PyTorch, Keras, Tensorflow libraries and train them with real-world datasets. • Design convolution networks for handwriting and object classification from images or video. • Design recurrent neural networks with attention mechanisms for natural language classification, generation, and translation. • Evaluate the performance of different deep learning models (e.g., with respect to the bias- variance trade-off, overfitting and underfitting, estimation of test error). • Perform regularization, training optimization, and hyperparameter selection on deep models.

Learning Outcome: Course Outcomes: On successful completion of the course, students will be able to: • Understand the mathematics behind functioning of artificial neural networks • Analyze the given dataset for designing a neural network-based solution • Carry out design and implementation of deep learning models for signal/image processing applications • Design and deploy simple TensorFlow-based deep learning solutions to classification problems

Syllabus:

Unit 1: Theory Credit: 3 [L 45]

Introduction to Deep Learning

Overview of deep learning: From machine learning to deep learning. History of deep learning, deep learning success stories. Gradient descent, stochastic gradient descent, momentum, and adaptive subgradient method. [L 5]

Neural Network

Overview, XOR problem, two-layer perceptrons. Architecture of multilayer feedforward network. Backpropagation algorithm for multilayer feedforward neural networks. [L 8]

Convolutional Neural Networks (CNN)

Overview of convolution, design and analysis of CNN, stacking, striding and pooling. Different variants of CNN: LeNet, AlexNet, ZF-Net, VGGNet, GoogLeNet, ResNet etc. [L 8]

Recurrent Neural Network (RNN)

Concept and applications. Backpropagation through time (BPTT), Vanishing and Exploding Gradients, Truncated BPTT, GRU, LSTMs, bidirectional LSTMs and their applications. [L 8]

Autoencoders

Undercomplete autoencoders, regularized autoencoders, sparse autoencoders, denoising autoencoders, representational power, layer, size, and depth of autoencoders. Sequence to Sequence Learning and Attention. Introduction to Transformers. Bidirectional Encoder Representations from Transformers (BERT). [L 8]

Deep Generative Models

Boltzmann Machines, Restricted Boltzmann Machines, Deep Belief Networks, Deep Boltzmann Ma-chines. [L 8]

Unit II: Deep Learning Lab Credit: 1 (L 30)

Practical will be conducted as per the topic covered in the theoretical parts.

Reading References:

1. Deep Learning and Neural Networks by Charu Aggarwal. ISBN: 978-3-319-94463-0.
2. Deep Learning by I. Goodfellow, Y. Bengio and A. Courville. ISBN: 978-0262-03561-3.
3. Neural Networks and Learning Machines by S. Haykin. ISBN: 978-0-13-14713-99.
4. Probabilistic Graphical Models: Principles and Techniques by D. Koller and N.Friedman. ISBN: 978-0262-01319-2.
5. Machine Learning by Tom Mitchell, McGraw-Hill, 1997. ISBN 0070428077.
6. Deep Learning From Scratch: Building with Python from First Principles by Seth Weidman published by O`Reilley.
7. Deep Learning with Python by Francois Chollet.
8. Advanced Deep Learning with Keras by Rowel Atienza.
9. Hands-On Deep Learning Algorithms with Python by Sudharsan Ravichandran.
10. Machine Intelligence: Demystifying Machine Learning, Neural Networks and Deep Learning by Suresh Samudrala.