Plans for week 46, November 10-14

Dear all, welcome back to FYS-STK3155/4155. We hope you had a great weekend and that project 2 is close to its finalization. We wish you all the very best with the finalization of the project.?

Today's lecture continues with deep learning methods. We start now with our penultimate topic, namely recurrent neural networks (RNNs), which have served as a precursor to LLMs as we know them today. It is also a popular method for learning time series. After RNNs, our last topic is going to be autoencoders. These two deep learning methods will be the topic of the last three lectures on November 10, 17 and 24. November 24 is our last lecture (note error in the UiO schedule, the 24th is missing). Project 3 is also available and we assigned a tentative deadline to December 15.?

We will discuss project 3 during the various lab sessions this week. The tentative plan for this week is thus:

Plan for week 46

Material for the lecture on Monday November 10, 2025.

Intro to and mathematics of Recurrent Neural Networks (RNNs)

Lecture notes at?https://github.com/CompPhysics/MachineLearning/blob/master/doc/pub/week46/ipynb/week46.ipynb

Lab sessions on Tuesday and Wednesday.

Work on and discussions of project 3, deadline December 15 at midnight. Project 3 can be found at?https://github.com/CompPhysics/MachineLearning/tree/master/doc/Projects/2025/Project3.?Note that possible minor revisions may be done before the lab sessions start on November 11.

Reading recommendations

For RNNs, see Goodfellow et al chapter 10, see?https://www.deeplearningbook.org/contents/rnn.html.

Reading suggestions for implementation of RNNs in PyTorch: see Rashcka et al.'s chapter 15 and GitHub site at?https://github.com/rasbt/machine-learning-book/tree/main/ch15.

TensorFlow examples

For TensorFlow (using Keras) implementations, we recommend

David Foster, Generative Deep Learning with TensorFlow, see chapter 5 at?https://www.oreilly.com/library/view/generative-deep-learning/9781098134174/ch05.html

Joseph Babcock and Raghav Bali Generative AI with Python and their GitHub link, chapters 2 and 3 at?https://github.com/PacktPublishing/Hands-On-Generative-AI-with-Python-and-TensorFlow-2

Best wishes for the coming week and the finalization of project 2.

Morten et al.

p.s. For those of you interested, one may skip the second lecture today and attend the lecture of Yoshua Bengio, one of the authors of the deep learning book we have been using (the best one on the topics before 2020). Bengio is the winner of the Turing medal (the Nobel prize in informatics) and is considered as one of the founders of modern ML/AI and in particular deep learning.? See https://kommunikasjon.ntb.no/pressemelding/18712828/ki-ens-gudfar-yoshua-bengio-apner-norges-nye-senter-for-palitelig-kunstig-intelligens?publisherId=7742191&lang=no for more info