Plans for week 40, September 30-October 4

Dear all, we hope you have passed a great weekend! Here are our tentative plans for week 40:

Plans for week 40

Lecture Monday September 30, 2024

1. Stochastic Gradient descent with examples and automatic differentiation. Continuation of the discussions from last week

2. If we get time, we start with the basics of Neural Networks, setting up the basic steps, from the simple perceptron model to the multi-layer perceptron model

Suggested readings and videos

Readings and Videos:

1. The lecture notes for week 40 (these notes), see for example the jupyter-notebook at https://github.com/CompPhysics/MachineLearning/blob/master/doc/pub/week40/ipynb/week40.ipynb (some of the material will be revised late Sunday Sept 29).

2. For a good discussion on gradient methods, we would like to recommend Goodfellow et al section 4.3-4.5 and sections 8.3-8.6. We will come back to the latter chapter in our discussion of Neural networks as well.

3. For neural networks we recommend Goodfellow et al chapter 6 and Raschka et al chapter 2 on the simple perceptron model and later (next week) chapter 11 on how to build a neural network. Chapter 2 of Raschka et al has also a discussion on gradient descent.

4. Video on gradient descent at https://www.youtube.com/watch?v=sDv4f4s2SB8

5. Video on stochastic gradient descent at https://www.youtube.com/watch?v=vMh0zPT0tLI

6. Neural Networks demystified at https://www.youtube.com/watch?v=bxe2T-V8XRs&list=PLiaHhY2iBX9hdHaRr6b7XevZtgZRa1PoU&ab_channel=WelchLabs

7. Building Neural Networks from scratch at URL:https://www.youtube.com/watch?v=Wo5dMEP_BbI&list=PLQVvvaa0QuDcjD5BAw2DxE6OF2tius3V3&ab_channel=sentdex"

Lab sessions Tuesday and Wednesday

Material for the active learning sessions on Tuesday and Wednesday.

• Work on project 1 and discussions on how to structure your report

• No weekly exercises for week 40, project work only

• Video on how to write scientific reports recorded during one of the lab sessions at https://youtu.be/tVW1ZDmZnwM

• A general guideline can be found at https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md.

Best wishes to you all,

Fahimeh, Ida, Karl Henrik, Mia, Morten, Odin, and Sigurd