Plans for week 41, October 7-11
Dear all, welcome back to a new week with FYS-STK3155/4155. We hope the finalization of the first project is going well. See also the email we sent about the final stages of project 1.
The plans for this week include
Lecture on Monday October 7, 2024
-
Neural Networks, setting up the basic steps, from the simple perceptron model to the multi-layer perceptron model.
-
Building our own Feed-forward Neural Network
- Project 2 is available Monday evening and will be discussed during the lab sessions
Readings and Videos:
-
Lecture notes at https://github.com/CompPhysics/MachineLearning/blob/master/doc/pub/week41/ipynb/week41.ipynbLinks to an external site.
-
Rashcka et al chapter 11
-
For neural networks we recommend Goodfellow et al chapter 6.
a. Neural Networks demystified at https://www.youtube.com/watch?v=bxe2T-V8XRs&list=PLiaHhY2iBX9hdHaRr6b7XevZtgZRa1PoU&ab_channel=WelchLabsLinks to an external site.
-
Building Neural Networks from scratch at https://www.youtube.com/watch?v=Wo5dMEP_BbI&list=PLQVvvaa0QuDcjD5BAw2DxE6OF2tius3V3&ab_channel=sentdexLinks to an external site.
-
Video on Neural Networks at https://www.youtube.com/watch?v=CqOfi41LfDwLinks to an external site.
-
Video on the back propagation algorithm at https://www.youtube.com/watch?v=Ilg3gGewQ5ULinks to an external site.
We also recommend Michael Nielsen's intuitive approach to the neural networks and the universal approximation theorem, see the slides at http://neuralnetworksanddeeplearning.com/chap4.htmlLinks to an external site..
Lab sessions on Tuesday and Wednesday
-
Exercise on writing your own stochastic gradient and gradient descent codes. This exercise continues next week with studies of automatic differentiation
-
One lecture at the beginning of each session on the material from weeks 39 and 40 and how to write your own gradient descent code
-
Discussion of project 2
-
Your task before the sessions: revisit the material from weeks 39 and 40 and in particular the material from week 40 on stochastic gradient descent