TensorFlow examples

It seemed to me while learning TensorFlow that many example codes are not as consistent or intuitive as they could be for beginners, often obscuring the shared structure of deep learning models. This is my attempt to remedy the situation. That said, I'm grateful to the authors of the original code for doing the hard part of getting things to work. I'm still learning, so please let me know if you have suggestions for making the code even clearer.

Notes

This code has been tested with Python 3.5 and TensorFlow 1.0.1. Many examples generate a figure or two so take a look at figs/ to see what to expect. Results should be consistent between runs on a single CPU, but not necessarily on a GPU. An "epoch" means one full presentation of the dataset.

Datasets

• datasets/iris/iris.txt, a subset of the full Iris flower dataset, is used to illustrate simple linear and logistic regression. It contains the sepal length, petal length, and species label for Iris versicolor and Iris virginica.

• The MNIST handwritten digit dataset downloads and/or extracts itself as needed.

• The Tiny Shakespeare corpus, which is originally from https://github.com/karpathy/char-rnn and consists of a subset of Shakespeare's works, is used for the char-rnn example.

• datasets/images/wsp_arch.png is a black-and-white photograph of Washington Square Park in New York City used for the random filter example.

Linear regression with the Iris dataset

• iris_linreg_np.py implements linear regression in NumPy with batch gradient descent, i.e., each gradient is computed on the entire dataset. It hardly needs to be said (but I'll say it anyway) that this is not a good way to do linear regression for real data.

• iris_linreg_tf.py does the same with TensorFlow.

• visualize_graph.py writes the computational graph for the linear regression loss to an event file so it can be visualized by running tensorboard --logdir=logs/visualize. Open the indicated IP address in a browser and click on the "Graphs" tab.

Binary logistic regression with the Iris dataset

• iris_logreg_np.py implements logistic regression in NumPy with stochastic gradient descent, i.e., each gradient is computed on one random example from the dataset.

• iris_logreg_tf.py does the same with TensorFlow.

Multiclass logistic regression with MNIST

• mnist_logreg.py implements logistic regression based on the code from https://www.tensorflow.org/get_started/mnist/beginners.

Convolutional neural network with MNIST

• filter.py demonstrates the effect of a convolution with a random filter on a black-and-white image.

• mnist_cnn.py implements a CNN based on the code from https://www.tensorflow.org/get_started/mnist/pros. Includes name scopes and code for displaying a list of trainable variables and counting the number of parameters in the model, which I find useful to know.

• mnist_cnn_model.py, mnist_cnn_train.py, and mnist_cnn_test together do the same with dropout in the fully connected layer, and demonstrate a more canonical way of organizing the code so that the model can be simultaneously used for training and extracting predictions. Includes variable scopes, saving and restoring checkpoints, and using TensorBoard to monitor training progress (for which, run tensorboard --logdir=logs/mnist in another terminal). Run python mnist_cnn_train.py to train, and python mnist_cnn_test.py to extract a prediction.

$ python mnist_cnn_train.py

Variables
---------
conv1/W:0 (5, 5, 1, 32)
conv1/b:0 (32,)
conv2/W:0 (5, 5, 32, 64)
conv2/b:0 (64,)
fc/W:0 (3136, 1024)
fc/b:0 (1024,)
softmax/W:0 (1024, 10)
softmax/b:0 (10,)
=> Total number of parameters = 3274634
After 1 epochs, validation accuracy = 0.9574000239372253
After 2 epochs, validation accuracy = 0.9782000184059143
After 3 epochs, validation accuracy = 0.9818000197410583
After 4 epochs, validation accuracy = 0.9832000136375427
After 5 epochs, validation accuracy = 0.9887999892234802
After 6 epochs, validation accuracy = 0.9891999959945679
After 7 epochs, validation accuracy = 0.9890000224113464
After 8 epochs, validation accuracy = 0.989799976348877
After 9 epochs, validation accuracy = 0.9900000095367432
After 10 epochs, validation accuracy = 0.989799976348877
After 11 epochs, validation accuracy = 0.9909999966621399
After 12 epochs, validation accuracy = 0.991599977016449
After 13 epochs, validation accuracy = 0.9909999966621399
After 14 epochs, validation accuracy = 0.9911999702453613
After 15 epochs, validation accuracy = 0.9926000237464905
After 16 epochs, validation accuracy = 0.9922000169754028
After 17 epochs, validation accuracy = 0.9927999973297119
After 18 epochs, validation accuracy = 0.9922000169754028
After 19 epochs, validation accuracy = 0.9926000237464905
After 20 epochs, validation accuracy = 0.9937999844551086
Test accuracy = 0.9924

Variational autoencoder (VAE) with MNIST

• vae.py implements a VAE based on the code from https://jmetzen.github.io/2015-11-27/vae.html. Note that the KL term is computed using Equation (10) in the original paper.

$ python vae.py

Variables
---------
encoder/hidden_1/W:0 (784, 500)
encoder/hidden_1/b:0 (500,)
encoder/hidden_2/W:0 (500, 500)
encoder/hidden_2/b:0 (500,)
encoder/mean/W:0 (500, 2)
encoder/mean/b:0 (2,)
encoder/log_var/W:0 (500, 2)
encoder/log_var/b:0 (2,)
decoder/hidden_1/W:0 (2, 500)
decoder/hidden_1/b:0 (500,)
decoder/hidden_2/W:0 (500, 500)
decoder/hidden_2/b:0 (500,)
decoder/reconstruction/W:0 (500, 784)
decoder/reconstruction/b:0 (784,)
=> Total number of parameters = 1289788
After 5 epochs, loss = 155.88664106889203
After 10 epochs, loss = 148.58128617720172
After 15 epochs, loss = 145.61274827436966
After 20 epochs, loss = 143.976779535467
After 25 epochs, loss = 142.8190042669123
After 30 epochs, loss = 142.0120587019487
After 35 epochs, loss = 141.4040315662731
After 40 epochs, loss = 140.78155033458364
After 45 epochs, loss = 140.41188788674094
After 50 epochs, loss = 139.98751409357246
After 55 epochs, loss = 139.62178749778053
After 60 epochs, loss = 139.27646353981712
After 65 epochs, loss = 139.04612417047673
After 70 epochs, loss = 138.74444517655806
After 75 epochs, loss = 138.50106091586025

Deep Recurrent Attentive Writer (DRAW) with MNIST

• draw.py implements DRAW, a recurrent VAE with attention, based on https://github.com/ericjang/draw. Note the basic similarity with the VAE code above.

Variables
---------
encoder/read/W:0 (256, 5)
encoder/read/b:0 (5,)
encoder/lstm_cell/weights:0 (562, 1024)
encoder/lstm_cell/biases:0 (1024,)
encoder/mean/W:0 (256, 10)
encoder/mean/b:0 (10,)
encoder/var/W:0 (256, 10)
encoder/var/b:0 (10,)
decoder/lstm_cell/weights:0 (266, 1024)
decoder/lstm_cell/biases:0 (1024,)
decoder/reconstruction/patch/W:0 (256, 25)
decoder/reconstruction/patch/b:0 (25,)
decoder/reconstruction/write/W:0 (256, 5)
decoder/reconstruction/write/b:0 (5,)
=> Total number of parameters = 864055
After 1 epochs, loss = 201.9442735706676
After 2 epochs, loss = 152.0951077825373
After 3 epochs, loss = 132.94116324684836
After 4 epochs, loss = 123.02201456243341
After 5 epochs, loss = 117.14258439497515
After 6 epochs, loss = 113.38466285011985
After 7 epochs, loss = 110.65857582785867
After 8 epochs, loss = 108.49950408935547
After 9 epochs, loss = 106.83435360995206
After 10 epochs, loss = 105.4043239662864
After 11 epochs, loss = 104.31813178322533
After 12 epochs, loss = 103.3717008001154
After 13 epochs, loss = 102.62253411032937
After 14 epochs, loss = 101.94451520052823
After 15 epochs, loss = 101.26953340010209
After 16 epochs, loss = 100.77957465431906
After 17 epochs, loss = 100.32966930042613
After 18 epochs, loss = 99.84818851817738
After 19 epochs, loss = 99.42697262850675
After 20 epochs, loss = 99.06217550104314

(The PNG files were turned into an animated GIF using ImageMagick with the command convert -delay 20 -loop 0 figs/draw/* assets/draw.gif)

char-rnn with Shakespeare

• char_rnn_reader.py, char_rnn_model.py, char_rnn_train.py, and char_rnn_test.py together implement an LSTM character-level language model based on https://github.com/sherjilozair/char-rnn-tensorflow. Includes RNN cells, dynamic_rnn, dropout for RNNs, gradient clipping, embeddings (and pinning to the CPU), and a demonstration of how to use the TensorFlow flag system for command-line arguments. The Reader class in char_rnn_reader.py is an interface to the data; each batch is a list of words (inputs) and the list of words that follow each of those words (targets), as you can see by running python char_rnn_reader.py on its own. Run python char_rnn_train.py to train, then try different start texts with char_rnn_test.py to see where the model takes you (but some samples are more plausible than others):

$ python char_rnn_test.py --start_text="The meaning of life is "
The meaning of life is service: and it
false ready to the liberal, in my meaning judgment
his resign unless sorrow from nothing:
All absent and you here to someware believe.

HENRY BOLINGBROKE:
See you besides, too worn; but a day will not.

KING EDWARD IV:
Richard, ho! Then all the troth,
When you kill'd, we still desire to England.

GLOUCESTER:
Are they better on him, make fear those that
did hear our old suitors for the sacrament,
Now I grief follow'd by the choices of marriage?

JULIET:
Go, spy up in reproved trai

Generative adversarial network (GAN) for a normal distribution

• normal_gan.py implements a GAN based on the code from http://blog.evjang.com/2016/06/generative-adversarial-nets-in.html and http://blog.aylien.com/introduction-generative-adversarial-networks-code-tensorflow/. Some of the tricks mentioned in those blog posts didn't seem to be necessary for this simple example so are not included. Demonstrates the use of an exponentially decaying learning rate and optimizing with respect to subsets of the model parameters.

Asynchronous advantage actor-critic (A3C) reinforcement learning for Atari

• a3c_*.py implement A3C for Atari games using the OpenAI Gym environment. It requires Gym, Universe, and OpenCV, which you can install, for example, by running

pip install "gym[atari]"
pip install universe
pip install opencv-python

For the most part, this is just a stripped-down version of the already excellent code at https://github.com/openai/universe-starter-agent. I hope it's a bit more readable and therefore easier to modify, but it's also not as general as the original code - for instance, it cannot play games over VNC and doesn't work for earlier versions of TensorFlow. Use a3c_train.py to train (note this uses tmux to launch processes by default). a3c_train.py calls a3c_worker.py to launch a parameter server called ps and num-workers workers that experience the environment. Use TensorBoard to monitor progress during training and a3c_test.py at any time to generate a video of the agent playing. Both examples use just 2 workers, but basically the more workers (with more cores) the better.

Note: If you are using a virtual environment you have to put source [path to activate] [environment name] in your .bashrc file so the correct environment is used when new shells are opened.

Pong:

$ python a3c_train.py --env-id=PongDeterministic-v3 --num-workers=2 --log-dir=/tmp/pong
$ tensorboard --logdir=/tmp/pong
$ python a3c_test.py --env-id=PongDeterministic-v3 --log-dir=/tmp/pong --movie-path=movies/pong

Breakout:

$ python a3c_train.py --env-id=BreakoutDeterministic-v3 --num-workers=16 --log-dir=/tmp/breakout
$ tensorboard --logdir=/tmp/breakout
$ python a3c_test.py --env-id=BreakoutDeterministic-v3 --log-dir=/tmp/breakout --movie-path=movies/breakout

(The original mp4 files were converted to animated GIFs with the command convert -delay 2 movies/breakout.mp4 assets/breakout.gif and similarly for Pong so they could be included here.)

Additional resources

• Deep convolutional generative adversarial network (DCGAN): https://github.com/carpedm20/DCGAN-tensorflow

• Evolution strategies (ES) for reinforcement learning: https://github.com/openai/evolution-strategies-starter