Skip to content

ibatra/nlm

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

16 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Neural Language Models

This repository contains neural language model implementations trained and tested on Penn Treebank.

  1. Multi-layer LSTM with Dropout: The link to the notebook is here. It receives perplexity around 80.6 on test set on default parameters.
  2. Gated Convolutional Networks with Residual Connections: The link to the notebook is here. It receives perplexity around 70.9 on test set on default parameters.

GCNN trains a lot faster than LSTM, due to stacked convolutions performaing parallely. However, this implementation is currently done for fixed word lengths. I am still unclear how to approach for variable lengths.

Requirements

You will need Pytorch 0.4 and Python 3.5 to run this.

How to run

  1. For LSTM code simply run like python3 rnn.py
  2. For GCNN code simply run like python3 gcnn.py

References

LSTM:

  1. Pytorch Language Model
  2. Offical Pytorch Tutorial on LSTM

GCNN:

  1. Language Modeling with Gated Convolutional Networks on arXiv
  2. Unofficial implementation 1 of GCNN
  3. Unofficial implementation 2 of GCNN

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published