MSBA6461-Advanced-AI

This repository contains lecture materials for MSBA 6461: Advanced AI for Natural Language Processing. It will be updated regularly as new materials are added. The most recent iteration of the course is Spring 2024.

Important Change: As a result of the overall MSBA curiculum revision, the course has been renamed to "Advanced AI for Natural Language Processing" (previously "Advanced AI for Business Applications"). The course content has also been updated to reflect the new design. Specifically, all reinforcement learning materials have been moved to a folder named RL (legacy) and will not be covered in the course. The new course will focus on NLP and language models.

Most recent updates (Last edit: summer 2024):

• Added notebooks 5 and 6 with more transformer and LLM content.

Topics Covered in this Course

Below is a list of topics covered in each notebook.

1. Text Preprocessing and Representation Learning (Notebook 1)
   • Introduction to Natural Language Processing
   • Basic Text Pre-Processing
   • Bag-of-Words Representation
   • Word Embeddings (Skip Gram and Continuous Bag-of-Words)
2. RNN for Text Classification (Notebook 2)
   • Recurrent Neural Networks (RNN)
   • Long-Term Dependency Problem
   • LSTM and GRU
   • Bidirectional RNN
3. Sequence to Sequence Model (Notebook 3)
   • The Encoder-Decoder Architecture
4. Attention, Transformer, and LLMs (Notebook 4)
   • Attention Mechanism
5. Transformer (Notebook 5)
   • Introduction to Transformer Architecture
   • Transformer Implementation with in pytorch
6. Large Language Model (Notebook 6)
   • Overview of popular architectures and training processes
   • Demo with BERT encoder

In addition to the notebooks,

• The data folder contains all the data used in the notebooks.
• The pytorch folder contains the pytorch implementation for NLP-related demonstrations.
• The .vscode folder contains display settings for VSCode.
• The RL (legacy) folder contains reinforcement learning materials that were previously covered in the course.

Software Setup

Other than Jupyter Notebook and Python, this course makes use of several machine learning / deep learning packages. I recommend creating a Python environment and install the packages specified in the requirement.txt file.