Deep Learning

Deep Learning is about learning multiple levels of representation and abstraction that help to make sense of data such as images, sound, and text. Deep Learning is a new area of Machine Learning research, which has been introduced with the objective of moving Machine Learning closer to one of its original goals: Artificial Intelligence. Deep learning architectures such as deep neural networks, deep belief networks and recurrent neural networks have been applied to fields including computer vision, speech recognition, natural language processing, audio recognition, social network filtering, machine translation and bioinformatics where they produced results comparable to and in some cases superior to human experts. Through this course, we will learn deep learning from basic concepts to high-level techniques, and apply deep learning techniques to resolve practical problems.

Course Goal

Through this course, the student will capture the basic concepts, theory and techniques of deep learning. Moreover, the students will have knowledge of the recent advances of deep learning. Lastly, the students will be required to use deep learning to resolve practical problems in industry. Specific requirements are as follows:

Readings

In the style of graduate seminars, I will expect you to have read required readings prior to class and to watch required videos prior to class. Come prepared to class to discuss the material (asking clarification questions, working through the math, relating papers to each other, critiquing the papers, presenting original ideas related to the paper).

Homework Assignments

We can all delude ourselves into believing we understand some math or algorithm by reading, but implementing and experimenting with the algorithm is both fun and valuable for obtaining a true understanding. Students will implement small-scale versions of as many of the models we discuss as possible. I will give 3 homework assignments that involve implementation over the semester, details to be determined. My preference is for you to work in PyTorch. Therefore, you are required to learn PyTorch before class. One or more of the assignments may involve writing a commentary on a research article or presenting the article to the class.

Semester Grades

Semester grades will be based 20% on class attendance and participation, 30% on the homework assignments and 50% of exams.

Preliminary knowledge

• Linear Algebra
• Probability Theory
• Machine Learning

Reference materials

• Deep Learning Tutorials: http://deeplearning.net/tutorial/
• PyTorch Tutorials: http://pytorch.org/tutorials/
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift: https://arxiv.org/abs/1502.03167
• ImageNet Classification with Deep Convolutional Neural Networks, NIPS, 2012.
• Deep Residual Learning for Image Recognition, CVPR, 2016: https://arxiv.org/abs/1512.03385