Neural Models for some NLP related tasks implemented using PyTorch
Following are some of the NLP related tasks that I have tried to implement using PyTorch:
Machine translation is the task of automatically converting source text in one language to text in another language. Neural machine translation, or NMT for short, is the use of neural network models to learn a statistical model for machine translation.
NMT generally uses an Encoder-Decoder RNN structure. To increase the performance, Decoders also use something known as Attention mechanism which enables them to learn to focus on certain portions of the input sentence in order to predict the correct output word in the sequence.
The NMT Model implemented by me uses an Encoder + Attention-based Decoder to try and translate English sentences to Hindi.
The dataset was obtained from Tab-delimited Bilingual Sentence Pairs & can be found here.
Here is the IPython Notebook for the project.
The trained models can be found here: Encoder & Decoder.
References:
- Neural Machine Translation by Jointly Learning to Align & Translate by Bahdanau, Cho & Bengio
- NLP from Scratch: Translation with a Sequence to Sequence Network & Attention
Semantic slot filling is one of the most challenging problems in spoken language understanding (SLU). Intent Detection and Slot Filling is the task of interpreting user commands/queries by extracting the intent and the relevant slots. The semantic parsing of input utterances in SLU typically consists of three tasks: domain detection, intent determination, and slot filling.
The task at hand is to annotate (tag) each word of a query whether it belongs to a specific item of information (slot), and which one.
The model that I have tried to implement is a recurrent model consisting of an embedding layer, a bidirectional GRU cell, and a dense layer to compute the posterior probabilities.
SNIPS is a dataset by Snips.ai for Intent Detection and Slot Filling benchmarking available from the github page. This dataset contains several day to day user command categories (e.g. play a song, book a restaurant). I have used a slightly pre-processed version of this dataset, which can be found here.
Here is the IPython Notebook for the project
The trained model can be found here.
References:
- Using Recurrent Neural Networks for Slot Filling in Spoken Language Understanding
- Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces
- CNTK 202: Language Understanding with Recurrent Networks
- Pad pack sequences for Pytorch batch processing with DataLoader
Created with ❤️ by Tezan Sahu