Plans for the week of March 2-6

Dear all welcome back to FYS5429/9429.

The plan this week is to finish our discussion of recurrent neural networks, with a focus on the mathematics and how to write an RNN code and long-short-term-memory RNNs. After this (next week) we will discuss the whys and hows of autoencoders, before w emake a quick dive into transformers and then start with generative methods.

Plans for the week of March 2-6

Reminder on basics of recurrent neural networks (RNNs)

Mathematics of RNNs

L(ong)S(hort)T(erm)M(emory)s and RNNs

Writing our own codes for RNNs

Lab session: discussion and work on projects

Reading recommendations:

a. Goodfellow, Bengio and Courville's chapter 10 from?Deep Learning

b.?Sebastian Rashcka et al, chapter 15, Machine learning with Sickit-Learn and PyTorch

c.?David Foster, Generative Deep Learning with TensorFlow, see chapter 5

The last two books have codes for RNNs in PyTorch and TensorFlow/Keras.

The jupyter-notebook for this week is at https://github.com/CompPhysics/AdvancedMachineLearning/blob/main/doc/pub/week7/ipynb/week7.ipynb

Best wishes to you all,

Morten, Oda and Ruben

p.s. Here's a cool GPT (from Egil Stavseth Furnes), see the mail from Egil here (Norwegian). If you cme across interesting software, applications and more, feel free to share!!

Hei Morten,?

Jeg ville bare dele noe jeg er superentusiastisk ovenfor og som er kjemperelevant for FYS5419/FYS5429, "Micrograd".

Det er kanskje den korteste implementeringen av GPT noensinne, skrevet av Andrej Karpathy (tidligere AI-sjef for Tesla og OpenAI)?

https://karpathy.ai/microgpt.html

https://karpathy.github.io/2026/02/12/microgpt/

https://gist.github.com/karpathy/8627fe009c40f57531cb18360106ce95

Kan godt hende du er kjent med det fra tidligere, men tenkte ? tipse om det uansett!

Fra nettsiden: "This is a brief guide to my new art project microgpt, a single file of 200 lines of pure Python with no dependencies that trains and inferences a GPT".

Beste hilsen,?

Egil?