Language Technology I

A Graduate Course

Info

Instructors: Dr. Jon Dehdari and Dr. Asad Sayeed
Class Location: (Former) CiP Room, building C7.2
Class Times: Lecture: Mondays 14:00-16:00 (c.t); Lab: Wednesdays 16:00-18:00 (s.t)
Class Dates: Oct. 31st - Feb. 15th
Jon's Offices: either room 1.15, building A2.2, or room 1.11 building D3.1
Asad's Office: room 3.04, building C7.4

Purpose

Language Technologies I teaches the theoretical foundation of modern computational linguistics and natural language processing. This includes important machine learning techniques.

Outline

1. Formal models of language: possibilities (homework)
2. Statistical models of language: probabilities (homework)
3. Applications of language models
4. n-gram language models and smoothing (more info) (homework) (training data) (testing data) (example transcript) (Bad-Turing transcript) (lazy tokenization script)
5. Parts of speech, word clusters, and class-based language models
6. Log-linear models (homework)
7. Word vectors, and applications (homework)
8. Feedforward neural networks and autoencoders (homework)
9. Recurrent neural networks and their language models (homework) (example transcript)
10. Probabilistic context-free grammars, parsing, and syntactic language models (homework - small updates)
11. Sequence-to-sequence models and neural machine translation
12. Convolutional networks and character-based models of language

External Links

• Neural Network Software:

• Keras, an easy neural network library. You can install it by typing the following from the command line:

  pip3 install --user keras
  

• List of other software for neural networks

• Another list of recurrent neural network stuff

• Free Corpora:

• WMT Data, both parallel and monolingual corpora

• ACL Wiki, "Resources by Language"

• OPUS - open parallel corpora

• Corpus Processing Tools:

• https://github.com/jonsafari/habeas-corpus

• https://github.com/kpu/preprocess

• http://corpus.tools