Natural Language Processing

Course Material for Natural Language Processing @ Computer Science Dept, Sapienza Master in Computer Science

Syllabus at a glance

NLP fundamentals and downstream tasks - Part I

Prof. Faralli

• Introduction to NLP, Regular Expressions, Finite State Automata and REs
• Words, Corpora and Text Normalization
• Spelling Correction and Minimum Edit Distance
• Language models, Part-of-speech-tagging
• Hidden Markov Model, Viterbi Algorithm, Logistic Regression
• Syntax, Semantics, Vector semantics (sparse), NLP tasks

Neural and multimodal NLP - Part II

Prof. Masi

• Latent Semantic Analysis and word2vec [hierarchical softmax & neg. sampling]
• Scaling word2vec, Sentiment Analysis, Language Model w/ Neural Nets
• Sequence modeling w/ Deep Learning: LM /w RNN, POS, Image Captioning
• from LSTM to Transformers
• Neural Machine Translation, Encoder/Decoder, Beam Search
• Contextual Embedding: BERT, GPT, Transfer Learning
• Multimodal NLP: Diffusion models (images), NLP as supervision for Vision (CLIP)
• text2image application (Dall-E 2): based on diffusion and CLIP

Material

📖 Course Material

It is in the form of Jupyter Notebook slides with LaTeX math, code, drawings, plots and explanations

• Slides and material will be uploaded before every lecture on Google Classroom and here.
• Good starting point but but may be not enough.
• Textbooks are required.

Material

Date	Topic	NBviewer	Github	Colab
Word embeddings
April 16, 18	LSA, intro word2vec
April 23, 30	scaling word2vec, Sentiment Analysis, Language Model w/ NN
April 30, May 2, 7	Deep learning for Seq. Processing
May 9, 14	From RNN to Neural Machine Translation
May 14, 21	Transformers, BERT, Pre-training
May 28	ExamPrep, CLIP