Syllabus

This is an overview of the mandatory readings for the exam. The syllabus generally consists of the lecture slides, weekly exercises, mandatory assignments, along with additional readings, described below.

Jurafsky & Martin 3rd ed. (August 2025 version):

• Chapter 2 (text normalization): sections 2.1, 2.4, 2.5, 2.6, 2.8 [Note: the slides refer to an earlier version of the book with a different section structure.]
• Chapter 3 (n-gram LMs): except 3.7
• Appendix B (Na?ve Bayes): except B.9, B.10
• Chapter 4 (logistic regression): except 4.15 [Note: there is substantial overlap between Appendix B and Chapter 4]
• Chapter 5 (vectors and embeddings): whole chapter
• Chapter 6 (neural networks): without the exact computations of 6.6
• Chapter 7 (large language models): whole chapter
• Chapter 8 (Transformers): except 8.3, 8.8, 8.9
• Chapter 9 (post-training and test-time compute): whole chapter
• Chapter 10 (masked language models): whole chapter
• Chapter 12 (machine translation): except 12.1, 12.4.1
• Chapter 14 (phonetics): only 14.1, 14.2 (but not the details of articulatory phonetics, just what we had covered in the lecture), 14.4 and 14.5 (but not the technical details of each step, just what we have covered in the lecture)
• Chapter 15 (ASR): only 15.1, 15.3 (not the technical details, but the intuition), 15.5 (just the intuition), and 15.6
• Chapter 16 (TTS): only 16.1, 16.2 (just the intuition)
• Chapter 17 (sequence labeling): except 17.7
• Chapter 25 (conversation and its structure)
• Appendix K (frame-based dialogue systems)

Other obligatory readings:

• On Zipf's law, collocations, type-token-ratio etc.:
  • Manning & Sch¨¹tze, Foundations of Statistical Natural Language Processing, chapter 1 (PDF on the website).
• On MDPs:
  • Section 24.6 from the?Dialogue chapter?of?2nd edition?of Jurafsky & Martin.
• On privacy:
  • Chapter 2?of Domingo-Ferrer, J., S¨¢nchez, D., & Soria-Comas, J. (2016).?Database anonymization: privacy models, data utility, and microaggregation-based inter-model connections.?Synthesis Lectures on Information Security, Privacy, & Trust,?8(1), 1-136.?NB: You can skip the technical details on measuring information loss.

Formulas:

We expect you to know the formulas listed below. However, the most important is to understand the logic behind them and to be able to explain how they should be applied and what they are used for.

• Zipf¡¯s laws, type-token ratio
• Language model training, additive smoothing, interpolation, perplexity
• Accuracy, precision, recall, F-measure, micro- and macro-averaging
• Bayes¡¯ theorem, Na?ve Bayes training and prediction formulas
• Softmax, logistic regression update rule
• HMM training formula, greedy inference formula
• Cosine similarity, TF-IDF weighting
• Sigmoid function, ReLU, cross-entropy loss
• Self-attention
• Bellman equation (and the definition of MDPs)
• Word error rate
• BLEU score
• Formulas for group fairness