Vehicle Detection Project
The goals / steps of this project are the following:
- Perform a Histogram of Oriented Gradients (HOG) feature extraction on a labeled training set of images and train a classifier Linear SVM classifier
- Optionally, you can also apply a color transform and append binned color features, as well as histograms of color, to your HOG feature vector.
- Note: for those first two steps don't forget to normalize your features and randomize a selection for training and testing.
- Implement a sliding-window technique and use your trained classifier to search for vehicles in images.
- Run your pipeline on a video stream and create a heat map of recurring detections frame by frame to reject outliers and follow detected vehicles.
- Estimate a bounding box for vehicles detected.
Rubric Points
###Here I will consider the rubric points individually and describe how I addressed each point in my implementation.
###Writeup / README
####1. Provide a Writeup / README that includes all the rubric points and how you addressed each one. You can submit your writeup as markdown or pdf. Here is a template writeup for this project you can use as a guide and a starting point.
You're reading it!
###Histogram of Oriented Gradients (HOG)
####1. Explain how (and identify where in your code) you extracted HOG features from the training images.
The code for this step is contained in the first section Explore the data and second section Feature selection of the IPython notebook.
I started by reading in all the vehicle and non-vehicle images. Here are some example images of the vehicle and non-vehicle classes:
I then explored different color spaces and different skimage.hog() parameters (orientations, pixels_per_cell, and cells_per_block). I grabbed random images from each of the two classes and displayed them to get a feel for what the skimage.hog() output looks like. I used skimage.hog() on gray, hls, and hsv channels. Below are some examples with HOG parameters of orientations=9, pixels_per_cell=(8, 8) and cells_per_block=(2, 2):
####2. Explain how you settled on your final choice of HOG parameters.
I tried various combinations of parameters and realize the default values (see above) from the course are actually pretty good. Eventually I choose to use HOG with these parameters on the hls channel of the images.
####3. Describe how (and identify where in your code) you trained a classifier using your selected HOG features (and color features if you used them).
I trained a linear SVM using the HOG output on the hls channle of the images as well as the color histogram and the bin spatials. A CalibratedClassifierCV is used to wrap around the LinearSVC in order to get the predict probability, instead of prediction of 0 or 1, to output. This can help to eliminate false positives in the pipeline. 20% of the training data is splitted to the test data, The linear SVM is able to achieve test accuracy of 0.9943.
###Sliding Window Search
####1. Describe how (and identify where in your code) you implemented a sliding window search. How did you decide what scales to search and how much to overlap windows?
I use three different sizes of windows to search 64 by 64, 96 by 96, and 128 by 128. For the y-axis, the search is limited only from 400 to 600 so we don't have to search the sky or the nose of the car. For the x-axis, we only search from 600 to 1280 as we don't have cars on the left side. This will only work for the current video. If we are not in the left most lane, we have to modify the values to cover the potential cars on the left side as well.
Below is an example image of my sliding windows.
####2. Show some examples of test images to demonstrate how your pipeline is working. What did you do to try to minimize false positives and reliably detect cars?
Ultimately I searched on three scales using HLS 3-channel HOG features plus spatially binned color and histograms of color in the feature vector, which provided a nice result. Here are some example images with their respective heatmaps:
####1. Provide a link to your final video output. Your pipeline should perform reasonably well on the entire project video (somewhat wobbly or unstable bounding boxes are ok as long as you are identifying the vehicles most of the time with minimal false positives.) Here's a link to my video result
####2. Describe how (and identify where in your code) you implemented some kind of filter for false positives and some method for combining overlapping bounding boxes.
I aggregated up to 5 consecutive heatmaps to get the average classification out to minimize false positives. The idea is usually the false positives do not appear in consecutive frames. After putting the consecutive heatmaps together, I was able to eliminate most of the false positives.
###Discussion
####1. Briefly discuss any problems / issues you faced in your implementation of this project. Where will your pipeline likely fail? What could you do to make it more robust?
The main issue right now is my pipeline runs very slow. Because I'm doing HOG transformations on all the sliding windows. The correct way as suggested by Ryan Keenan is to do one HOG transformation on the entire image and then do sliding windows on the HOG image. The implementation is far more complicated and I will try to get it done in the future update.