Plans for week 41

Dear all, we hope you have had a great weekend. The plans for this week are (although the deadline for project 1 has been extended to October 15)

Plans for week 41

Active learning sessions on Tuesday and Wednesday

• Exercise on writing your own stochastic gradient and gradient descent codes. This exercise continues next week with studies of automatic differentiation and will be used in project 2
• One lecture at the beginning of each session on the material from weeks 39 and 40 and how to write your own gradient descent code
• Discussion of project 2
• Your task before the sessions: revisit the material from weeks 39 and 40 and in particular the material from week 40 on stochastic gradient descent

Material for the lecture on Thursday October 12, 2023

• Neural Networks, setting up the basic steps, from the simple perceptron model to the multi-layer perceptron model.
• Building our own Feed-forward Neural Network
• Readings and Videos:
  • The lecture slides
  • For neural networks we recommend Goodfellow et al chapter 6.
  • Neural Networks demystified
  • Building Neural Networks from scratch
  • Video on Neural Networks
  • Video on the back propagation algorithm

We also recommend Michael Nielsen's intuitive approach to the neural networks and the universal approximation theorem, see the slides at http://neuralnetworksanddeeplearning.com/chap4.html.

Best wishes to you all,

Adam, Daniel, Fahimeh, Ida, Karl Henrik, Mia and Morten