Plans for week 39, September 23-27
Dear all and welcome to a new week with machine learning and much more.
This week we continue our discussions of logistic regression as our first encounter on classification methods. As we discussed during the lecture last week, we use logistic regression in order to introduce classification problems as well as gradient methods in order to find the optimal parameters of our model.
Important note: Our lectures from Monday September 23 and for the rest of the semester will be at Store Fysiske Lesesal, where we met the first time. The audio equipment has now been installed.
The plan this week (and we will continue with these topics next week as well) is:
Lecture Monday September 23
Material for the lecture on Monday September 23.
-
Repetition of Logistic regression equations and classification problems and discussion of Gradient methods. Examples on how to implement Logistic Regression and discussion of gradient methods
-
Stochastic Gradient descent with examples and automatic differentiation (theme also for next week). The lecture material at https://github.com/CompPhysics/MachineLearning/blob/master/doc/pub/week39/ipynb/week39.ipynb will also be discussed next week.
-
Readings and Videos:
-
Lecture notes at https://github.com/CompPhysics/MachineLearning/blob/master/doc/pub/week39/ipynb/week39.ipynb
-
For a good discussion on gradient methods, we would like to recommend Goodfellow et al section 4.3-4.5 and sections 8.3-8.6. We will come back to the latter chapter in our discussion of Neural networks as well.
-
Raschka et al, pages 53-76 on Logistic regression and pages 37-52 on gradient optimization
-
Video on gradient descent see https://www.youtube.com/watch?v=sDv4f4s2SB8
-
Video on stochastic gradient descent, https://www.youtube.com/watch?v=vMh0zPT0tLI
-
Lab sessions week 39
Material for the active learning sessions on Tuesday and Wednesday.
-
Discussions on how to structure your report for the first project
-
Exercise for week 39 on how to write the abstract and the introduction of the report and how to include references.
-
Work on project 1, in particular resampling methods like cross-validation and bootstrap. For more discussions of project 1, chapter 5 of Goodfellow et al is a good read, in particular sections 5.1-5.5 and 5.7-5.11.
- A general guideline can be found at https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md.
Best wishes to you all,
Fahimeh, Ida, Karl Henrik, Mia, Morten, Odin, and Sigurd
p.s. see also the attached notebook from Karl Henrik. This is a code example which shows the bias variance tradeoff, 10 models are trained with resampled training data and shown in a plot with the test data