Weekly plans and update for week 38
Hi all, here comes our weekly mailing (late this week, sorry for this) with updates and summaries.
Last week we discussed and derived the expressions for shrinkage methods like Ridge and Lasso and we made an analysis in terms of the singular value decomposition of a matrix and a Bayesian approach.
We showed also that the matrix XTX is proportional to the covariance of the same matrix. This meant that for example Ridge Regression shrinks those singular values which are smallest the most (or those which have the smallest covariance). The consequence, since the variance of β is proportional to the inverse of this matrix, is that for Ridge Regression shrinks 'away' attempts at reducing the variance of those $$\beta$ values that have a large variance.
Lasso does a similar job.
This week we will use Thursday to summarize these topics and link them to a discussion of Project 1. We will also discuss again the resampling methods like cross-validation and bootstrap and link these again with the tasks in project 1.
Most of Thursday will thus be devoted to a summary of ordinary least squares, Ridge and Lasso regression and how we are going to use these methods in connection with the project.
Thereafter we start (most likely towards the end of Thursday/begin of Friday) with logistic regression and classification.
The recommended reading is Hastie et al chapter 3.4, 3.6, 3.7 for Ridge, OLS and Lasso. For Cross-validation and Bootstrap, see chapters 7.10 and 7.11. For the bias-variance discussion, see chapters 7.1-7.6
For logistic regression on Friday see chapters 4.1, 4.2 and 4.4.
I have also changed the layout of the teaching material at https://compphysics.github.io/MachineLearning/doc/web/course.html. You will now find weekly slides, and for this week go to the html or jupyter-notebooks for week 38.
I am revising the material considerably, so stay tuned. I am still adding material to this week's slides. For weeks 34-37, I have also included the videos to the lectures and the handwritten notes.
The lecture notes for the first 6 weeks will also be revised accordingly and be linked at the same page where you find the projects and slides (pdf and jupyter-notebook formats).
Else, on Friday there will be only a recorded lecture (available tomorrow) due to me having to attend a meeting in person. I apologize that.
Best wishes to you all
Morten et al