Plans for second week, lecture January 30

Dear all, we hope this week has started the best possible way. Here are our plans for the lecture on January 30.

The aim this week is to give a review of the basics of neural networks. Many of you have seen similar material before, but we think it is useful to repeat some of the basics as neural networks are essential parts of most algorithms we describe later, whether these are CNNs, RNNs, autoencoders or other methods we will discuss.

We will also present and discuss different project variants. Next week we will also try to have presentations from those of you who have defined specific and own projects.

Note also that I have changed the zoom link to my UiO account. Our permanent zoom link for the rest of the semester is

https://uio.zoom.us/my/mortenhj

The jupyter-notebook  with the material for this week (with code examples) is at https://github.com/CompPhysics/AdvancedMachineLearning/blob/main/doc/pub/week2/ipynb/week2.ipynb

The plan for tomorrow is as follows:

Overview of second week, Lecture January 30

1. Mathematics of neural networks

2. Writing own code (bring back to life your NN code if you have one)

3. Discussion of project alternatives

4. Permanent Zoom link for the whole semester is https://uio.zoom.us/my/mortenhj

Videos on Neural Networks

• Video on Neural Networks at https://www.youtube.com/watch?v=CqOfi41LfDw

• Video on the back propagation algorithm at https://www.youtube.com/watch?v=Ilg3gGewQ5U

Mathematics of deep learning

Two recent books online.

1. The Modern Mathematics of Deep Learning, by Julius Berner, Philipp Grohs, Gitta Kutyniok, Philipp Petersen, published as Mathematical Aspects of Deep Learning, pp. 1-111. Cambridge University Press, 2022

2. Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory, Arnulf Jentzen, Benno Kuckuck, Philippe von Wurstemberger

Reminder on books with hands-on material and codes

• Sebastian Rashcka et al, Machine learning with Sickit-Learn and PyTorch

• David Foster, Generative Deep Learning with TensorFlow

• Bali and Gavras, Generative AI with Python and TensorFlow 2

All three books have GitHub addresses from where one can download all codes. We will borrow most of the material from these three texts as well as from Goodfellow, Bengio and Courville's text Deep Learning

Reading recommendations

1. Rashkca et al., chapter 11, jupyter-notebook sent separately, from GitHub

2. Goodfellow et al, chapter 6 and 7 contain most of the neural network background.

We will have our regular lab session right after the lecture. 

Best wishes,

Edvin and Morten