CMSC 475/675: Neural Networks

Instructor: Tejas Gokhale (OH: WED 1430--1530 or by appointment; ITE 214);
Teaching Assistant: TDB TBD;
Time: MON and WED 1600--1715
Location:

Course description

This class will offer a comprehensive overview of neural networks and deep learning algorithms. Deep Learning has been a highly successful research field over the last 20 years across a range of domains (vision, language, audio, robotics; ``AI'' in general). Deep learning has lead to significant commercial success and exciting new directions that may previously have seemed out of reach. The class will focus on the core principles of extracting meaningful representations from high-dimensional data, a fundamental aspect for several applications in autonomous decision making. Class lectures will cover fundamental topics such as network design, training and optimization, and evaluation. Homeworks will give students the opportunity to implement algorithms learnt in class for applications in visual recognition, language understanding and other domains. In the term project, students will construct a research hypothesis, propose new techniques and solutions, interpret results, and communicate key findings.

Prerequisites: We will assume that you have a basic (but solid) expertise in linear algebra, geometry, probability, and Python programming. Recommended classes at UMBC are: MATH 221 (Linear Algebra), STAT 355 or CMPE 320 (Probability and Statistics), MATH 151 (Calculus and Analytical Geometry). If you are unfamiliar with linear algebra or calculus, you should consider taking both: without these tools, you are likely to struggle with the course. Although we will provide brief math refreshers of these necessary topics, CMSC 475/675 should not be your first introduction to these topics.

Schedule

Schedule is tentative and subject to change.

	Topic	Resources	Optional Reading
1	Introduction	[slides]
2	Machine Learning Review	[slides]
3	Multi Layer Perceptron	[slides]
4	NN Training	[slides]
5	Convolutional Neural Networks	[slides]
6	Autoencoders	[slides]
7	Self-Supervised Learning	[slides]
8	Language Models	[slides]
9	Vision Transformers	[slides]
10	Autoencoders	[slides]
11	Multimodal Deep Learning	[slides]
12	Generative Adversarial Networks	[slides]
13	Diffusion Models	[slides]
14	Implicit Neural Representations	[slides]
15	Neural Radiance Fields	[slides]
16	Neural Operators	[slides]
17	Distributional Robustness	[slides]
18	Adversarial Robustness	[slides]

Grading

Please consult the syllabus for details.

Homework: 35%
Midterm: 20%
Quizzes: 10%
Project: 35%
Extra Credit: (max) 10%

The class has a mix of PhD, MS, and BS students. We believe that anyone with the above prerequisites and a will to learn will do well. Work hard, engage and participate in class, learn how to read, write, and present research articles, be creative in your projects, and seek help when needed!

Late Submission Policy

Everyone in this course has 7 late days to use as needed for personal reasons and emergencies. Do not use them as an excuse to procrastinate -- start working on your assignments early. Each late day extends the deadline by 24 hours and does not influence the grade. The late days can be used for homeworks only. Late submissions turned in after all 7 late days have been exhausted will not be evaluated and will receive 0 points. One-off extensions or exceptions will not be granted. I reserve the right to issue class-wide deadline extensions.

Projects

Projects will be judged on the basis of relative growth (from where you start to where you end). Grad projects should have an original and unique research hypothesis with a potential for publication. Undergrads students can also propose original and unique research hypothesis, but will be allowed to work on an idea provided by the instructors (i.e. you get to skip “ideation”) or innovative applications or combination of existing work.

BS or MS (coursework) students: I recommend working in groups of 3 students.
PhD students: Consult with Tejas during Office Hours and discuss your existing research agenda. We will integrate the course project into that agenda if possible. Group sizes (or individual projects) will be decided on a case-by-case basis.

Proposal: Clearly state the following:
- Problem you wish to tackle (and why)
- Proposed approach and methods
- Timeline
- What each student in the group will do.
- Expected Outcome and Worst-Case Outcome

Resources

CVPR Template: [link]
For a quick tutorial on LaTeX, visit: this Overleaf Tutorial.

Academic Integrity

Please read UMBC's policy on Academic Integrity. I take academic integrity seriously. I hope that we will never have to deal with violations -- they are never pleasant for anyone involved. Please read the policies stated in the Syllabus .