Instructor: Tejas Gokhale (OH: WED 1430--1530 ; ITE 214);
Teaching Assistant: Ziwei Zhang (OH: MON 1300--1400 and TUE 1430--1530; ITE 334)
Time: MON and WED 1600--1715
Location: ITE 231
This class will offer a comprehensive overview of neural network architectures and deep learning algorithms. Deep Learning has been a highly successful research field over the last 20 years across a range of domains (vision, language, audio, robotics; ``AI'' in general) and has also translated into significant commercial success. The class will focus on the core principles of extracting meaningful representations from high-dimensional data, a fundamental aspect for several applications in autonomous decision making. Class lectures will cover fundamental topics such as network design, training and optimization, and evaluation. Homeworks will give students the opportunity to implement algorithms learnt in class for applications in visual recognition, language understanding and other domains. In the term project, students will construct a research hypothesis, propose new techniques and solutions, interpret results, and communicate key findings.
Prerequisites: We will assume that you have a basic (but solid) expertise in linear algebra, geometry, probability, and Python programming. Recommended classes at UMBC are: MATH 221 (Linear Algebra), STAT 355 or CMPE 320 (Probability and Statistics), MATH 151 (Calculus and Analytical Geometry). If you are unfamiliar with linear algebra or calculus, you should consider taking both: without these tools, you are likely to struggle with the course. Although we will provide brief math refreshers of these necessary topics, CMSC 475/675 should not be your first introduction to these topics.
Schedule is tentative and subject to change.
| Topic | Resources | Optional Reading | |
| 0 | Introduction | [slides] | |
| 1 | Learning | [slides] | DL Book Ch 5 Sec 5.1. |
| 2, 3 | MLP and Gradient Descent | [slides_A] [slides_B] | DL Book Ch 6 |
| 4 | Convolutional Neural Networks | [slides] | DL Book Ch 9; Bishop DL Book Ch.10 |
| 5 | Training NNs | [slides] | |
| 6 | NN Generalization | ||
| 7 | Representation Learning | ||
| 8 | Generative Models | ||
| 9 | Transformers | ||
| 10 | Robustness | ||
| 11 | Uncertainty and Calibration | ||
| 12 | Implicit Neural Representations | ||
| 13 | Neural Operators |
Resources