Week 7: Feb 24-March 3

Interactive session, Thursday March 3

• Quiz
• Jupyter notebook: interactive_lin_reg_with_pizzas (updated March 8)
• Recording

Weekly lecture:

Slides

Video Recordings:

1. Linear Regression and Classification
2. The Logistic Function and its Derivative
3. The Logistic Regression Classifier
4. Cross-Entropy Loss
5. Training the Logistic Regression Classifier
6. Variants of Gradient Descent
7. Multi-Class Classification: one-vs.-rest
8. Multinomial Logistic Regression

Readings:

Paolo Perrotta, Programming Machine Learning (in O'Reilly library)

• Ch. 5 "A discerning Machine"
• Ch. 7 "The Final Challenge

Marsland does not cover all the stuff we have considered this week, in particular not Logistic Regression, Multinomial Logistic Regression and loss functions. Perrotta's book, ch. 5 has a useful dicussion of logistic regression and loss functions, while ch. 7 presents one-vs.-rest multi-class classification. Ch. 6 "Getting Real" presents the MNIST dataset to be used in ch. 7. We will also make use of MNIST later in the semester.

Observe, however, that Marsland, Sec. 4.6, "Deriving Back-Propagation", presents some of the same material as the lecture in the context of multi-layer neural networks (e.g., sec. 4.6.2, 4.6.3, 4.6.5, 4.6.6).

Weekly exercises

• Exercises
• Solutions