Week 13: Training neural nets

DATA 442/642 · Week 13

Module 6 — Training neural nets

Notes

• 13.1 Back-Propagation Challenges(slides)
• 13.2 Faster Optimizers(slides)
• 13.3 Hyperparameters(slides)
• 13.4 Regularization(slides)

Additional

• Back-propagation(PDF)

Discussion

• Name one reason a ReLU helps with vanishing gradients.
• Dropout versus weight decay: which one is stochastic at train time?