COMPSCI 682 Neural Networks: A Modern Introduction

Note

  • This is a tentative class outline and is subject to change throughout the semester.
  • Every Monday the lectures for the week will be posted. The videos for the previous week will be taken down when new videos are posted, but the slides will remain available. The reason we take down the videos is so that people do keep up with the videos.
  • One lecture is associated with Tuesday, and the other one with Thursday.
Event TypeDateDescriptionVideo LecturesCourse Materials
Lecture 1A,1B Tuesday, Jan. 25 1A: Course Organization and Structure
1B: Intro to Deep Learning, historical context.		 [Lecture 1A]
[Lecture 1B]
 [slides]
[slides]
[python/numpy tutorial]
[jupyter tutorial]
Lecture 2 Thursday, Jan. 27 Image classification and the data-driven approach
k-nearest neighbor
Linear classification		 [Lecture 2] [slides]
[image classification notes]
[linear classification notes]
Optional Zoom Discussion Monday, Jan. 31
Session 1: 9:00AM - 9:45AM EST
Session 2: 4:00PM - 4:45PM EST		 Python setup, Google Colab, and the basics of Python
Lecture 3 Tuesday, Feb. 1 Loss Functions
Optimization		 [Lecture 3]
Note: Lecture starts from time stamp [4:45]. Part of the video before that is not relevant for the current semester.		 [slides]
Lecture 4 Thursday, Feb. 3 Backpropagation & Neural Networks I [Lecture 4] [slides]
[backprop notes]
[Efficient BackProp] (optional)
related: [1], [2], [3] (optional)
Lecture 5 Tuesday, Feb. 8 Neural Networks II
Higher-level representations, image features
Vector, Matrix, and Tensor Derivatives 		[Lecture 5]
Note: The help session for the Chain rule will be on Monday, February 14, not on Friday, as it says in the video.		 [slides]
handout 1: Vector, Matrix, and Tensor Derivatives
handout 2: Derivatives, Backpropagation, and Vectorization
Deep Learning [Nature] (optional)
Lecture 6 Thursday, Feb. 10 Neural Networks III
Training Neural Networks I
Activation Functions 		[Lecture 6]
Note:Please ignore the “Logistics” about Assignment 1 mentioned in the video, as they pertain to last semester.		 [slides]
[Neural Nets notes 1] tips/tricks: [1], [2] (optional)
Optional Zoom Discussion Monday, Feb. 14
Session 1: 8:00AM - 8:45AM EST
Session 2: 2:00PM - 2:45PM EST		 Reviewing the chain rule, applying the chain rule to vectors
Lecture 7 Tuesday, Feb. 15 Training Neural Networks II
weight initialization, batch normalization 		[Lecture 7]
Note: Lecture starts from [ 3:02 ]. Discussion regarding projects from [ 3:02 - 22:47 ] is the same this semester. However the dates do not apply.		 [slides]
[Neural Nets notes 2]
[Batch Norm]
Copula Normalization (optional)
Lecture 8 Thursday, Feb. 17 Training Neural Network III:
babysitting the learning process, hyperparameter optimization 		[Lecture 8] [slides]
[Bengio 2012] (optional)
Lecture 9 Thursday, Feb. 24 Training Neural Network IV:
model ensembles, dropout 		[Lecture 9] [slides]
[Neural Nets notes 3]
LeNet (optional)
Lecture 10 Tuesday, Mar. 1 DropOut... continued [Lecture 10] [slides]
Lecture 11 (a) Thursday, Mar. 3 Convolutional Neural Networks [Lecture 11 with slides] [slides]
Lecture 11b Tuesday, Mar. 8 Convolutional Neural Networks... continued [Lecture 11b] [slides]
Lecture 11c Tuesday, Mar. 8 Convolutional Neural Networks... continued [Lecture 11c] [slides]
Lecture 11
(b) (c)		 Tuesday, Mar. 8 Convolutional Neural Networks (continued) [Lecture 11 (b) and (c)] [slides]
Optional Zoom Discussion Wednesday, Mar. 23
Session 1: 9:00AM - 9:45AM EST
Session 2: 4:00PM - 5:00PM EST		 Batch normalization
Lecture 12 (a) Tuesday, Mar. 22 ConvNets for spatial localization
Object detection 		[Lecture 12 (a)] [slides]
ResNet (optional)
FCN (optional)
Lecture 12 (b) Thursday, Mar. 24 ConvNets for spatial localization
Object detection (continued) 		[Lecture 12 (b)] [slides]
Lecture 14 (a) Tuesday, Mar. 29 Understanding and visualizing Convolutional Neural Networks, Plus Self-supervision [Lecture 14 (a)] [slides]
Visualization notes
Lecture 14 (b) Thursday, Mar. 31 Understanding and visualizing Convolutional Neural Networks, Plus Self-supervision (continued) [Lecture 14 (b)] [slides]
Lecture 15 Tuesday, April 5 Neural Texture Synthesis and Style Transfer
Creating Adversarial Examples 		[Lecture 15] [slides]
[slides]
Lecture 16 Thursday, April 7 Generative Adversarial Networks [Lecture 16] [slides]
[Style-GAN]
[Alias-Free GAN]
Lecture 16(RNN) Tuesday, April 12 Recurrent Neural Networks [Lecture 16: RNN ] [slides]
DL book RNN chapter (optional)
min-char-rnn, char-rnn, neuraltalk2
The Unreasonable Effectiveness of RNN (optional)
Understanding LSTM Networks (optional)
Lecture 16(RNN)(2) Thursday, April 14 Recurrent Neural Networks (continued) [Lecture 16: RNN (continued)] [slides]
Lecture Tuesday, April 19 Exam Review and Revisiting Neural Texture Synthesis and Style Transfer [Lecture 15 Revisited]
Lecture 17 Thursday, April 21 Word Embeddings [Lecture 17: Word Embeddings] [slides]
Midterm Tuesday, April 26 Mid-Term Review [Review]