Plans for week 41, October 6-10

Dear all, first of all thx so much for heroic efforts with project 1. We are truly impressed by what you have been doing. Keep up the good work and best wishes to you all with the finalization of project 1.

This week we start discussing how to actually develop a neural network code. This will be the topic for the second project.  We will make the project  available next week and discuss it in more detail during the lectures and the lab sessions. This week we plan to start with a simpler set of exercises where you implement the feed-forward part of a code for a neural network. The exercises for this week can then in turn be used as a basis for the code in project 2. 

The plans this week are (see also links to various videos):

Material for the lecture on Monday October 6, 2025

1. Neural Networks, setting up the basic steps, from the simple perceptron model to the multi-layer perceptron model.

2. Building our own Feed-forward Neural Network, getting started

Readings and Videos:

1. These lecture notes

2. For neural networks we recommend Goodfellow et al chapters 6 and 7.

3. Rashkca et al., chapter 11, jupyter-notebook sent separately, from GitHub

4. Neural Networks demystified at https://www.youtube.com/watch?v=bxe2T-V8XRs&list=PLiaHhY2iBX9hdHaRr6b7XevZtgZRa1PoU&ab_channel=WelchLabs

5. Building Neural Networks from scratch at https://www.youtube.com/watch?v=Wo5dMEP_BbI&list=PLQVvvaa0QuDcjD5BAw2DxE6OF2tius3V3&ab_channel=sentdex

6. Video on Neural Networks at https://www.youtube.com/watch?v=CqOfi41LfDw

7. Video on the back propagation algorithm at https://www.youtube.com/watch?v=Ilg3gGewQ5U

8. We also recommend Michael Nielsen's intuitive approach to the neural networks and the universal approximation theorem, see the slides at http://neuralnetworksanddeeplearning.com/chap4.html.

Mathematics of deep learning

Two recent books online.

1. The Modern Mathematics of Deep Learning, by Julius Berner, Philipp Grohs, Gitta Kutyniok, Philipp Petersen, published as Mathematical Aspects of Deep Learning, pp. 1-111. Cambridge University Press, 2022

2. Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory, Arnulf Jentzen, Benno Kuckuck, Philippe von Wurstemberger

Reminder on books with hands-on material and codes

Sebastian Rashcka et al, Machine learning with Sickit-Learn and PyTorch

Lab sessions on Tuesday and Wednesday

Aim: Getting started with coding neural network. The exercises this week aim at setting up the feed-forward part of a neural network.

Best wishes to you all,

Morten et al