End-to-End-Learning-for-Self-Driving-Cars

Introduction

This project is a tensorflow implementation of End to End Learning for Self-Driving Cars. It trains an convolutional neural network (CNN) to learn a map from raw images to sterring command. And it implements a method called VisualBackProp to visualize the contribution of each pixel of the input image.

Requirements

• Tensorflow >= r0.14
• opencv, numpy

Howto

• Download the dataset
• Split the dataset: python split_data.py

✗ python split_data.py -h
usage: split_data.py [-h] [--data_dir DATA_DIR] [--seed SEED]
                     [--train_prop TRAIN_PROP]
                     [--validation_prop VALIDATION_PROP]

optional arguments:
  -h, --help            show this help message and exit
  --data_dir DATA_DIR   Directory of data
  --seed SEED           random seed to generate train, validation and test set
  --train_prop TRAIN_PROP
                        The proportion of train set in all data
  --validation_prop VALIDATION_PROP
                        The proportion of validation set in all data

• Train the model: python train.py

✗ python train.py -h
usage: train.py [-h] [--max_steps MAX_STEPS] [--print_steps PRINT_STEPS]
                [--learning_rate LEARNING_RATE] [--batch_size BATCH_SIZE]
                [--data_dir DATA_DIR] [--log_dir LOG_DIR]
                [--model_dir MODEL_DIR] [--disable_restore DISABLE_RESTORE]

optional arguments:
  -h, --help            show this help message and exit
  --max_steps MAX_STEPS
                        Number of steps to run trainer
  --print_steps PRINT_STEPS
                        Number of steps to print training loss
  --learning_rate LEARNING_RATE
                        Initial learning rate
  --batch_size BATCH_SIZE
                        Train batch size
  --data_dir DATA_DIR   Directory of data
  --log_dir LOG_DIR     Directory of log
  --model_dir MODEL_DIR
                        Directory of saved model
  --disable_restore DISABLE_RESTORE
                        Whether disable restore model from model directory

• Visualize your training procedure: tensorboard --logdir=./logs
• Test on the test set: python test.py

✗ python test.py -h
usage: test.py [-h] [--data_dir DATA_DIR] [--model_dir MODEL_DIR]

optional arguments:
  -h, --help            show this help message and exit
  --data_dir DATA_DIR   Directory of data
  --model_dir MODEL_DIR
                        Directory of saved model

• Find the salient objects

✗ python visulization.py -h
usage: visualization.py [-h] [--model_dir MODEL_DIR] [--data_dir DATA_DIR]
                        [--result_dir RESULT_DIR]
                        [--visualization_num VISUALIZATION_NUM]

optional arguments:
  -h, --help            show this help message and exit
  --model_dir MODEL_DIR
                        Directory of saved model
  --data_dir DATA_DIR   Directory of data
  --result_dir RESULT_DIR
                        Directory of visualization result
  --visualization_num VISUALIZATION_NUM
                        The image number of visualization

Training Results

The model structure visualized by tensorboard:

The curve of training loss:

Test results

Performance On Test

Loss (MSE) in test dataset: 0.016554169347
MAE in test dataset:  0.0626648643461

Visualization

Examples 1

Original image	Mask	Overlay

Examples 2

Original image	Mask	Overlay

Examples 3

Original image	Mask	Overlay

Examples 4

Original image	Mask	Overlay

Acknoledgements

Thanks to Sully Chen for the dataset.