Week 2

Lecture 2: Words, tokenization, tagged text

This week we will look at

  • some basic linguistics concepts related to words
  • the processes of tokenization and normalization
  • tagged text

Slides

Recording

Mandatory reading

Jurafsky and Martin, Speech and Language Processing, 3. ed. (edition of 16 Oct,. 2019!)

  • Ch. 2 Regular expressions etc
    • Sec. 2.0
    • Sec. 2.2 Words
    • Sec. 2.3 Corpora
    • Sec. 2.4 Normalization, except 2.4.3 and the technical details of 2.4.1
  • Ch. 8 Part-of-spee