Skip to content

Jetson Nano application that detects and tracks vehicles in a roundabout with Yolov4, DeepSort and NvDCF using DeepStream SDK and sends the info to a Kafka message bus.

Notifications You must be signed in to change notification settings

alxandru/vehicle_tracking_deepstream

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

19 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Vehicle Detection and Tracking with Yolov4 and DeepStream SDK

Alt Text

This application runs on Jetson Nano and detects and tracks vehicles in a roundabout from a fixed camera stream using Nvidia DeepStream SDK and sends information about the entries and the exits along with the vehicle ids to a Kafka Message Bus in order for a client application to process the data.

For detection the application uses a custom trained Yolov4-Tiny network based on RoundaboutTraffic dataset. DeepStream-Yolo was used to improve inference performance.

For tracking the Discriminative Correlation Filter tracker as well as the DeepSORT tracker were tested and used.

A Kafka producer based on librdkafka library was implemented to send information about the tracked vehicles to the message bus.

Table of contents

Requirements

Installation

Once we have all the requirements installed we download the repo and the third parties:

$ git clone https://github.com/alxandru/vehicle_tracking_deepstream.git
$ cd vehicle_tracking_deepstream
$ git submodule update --init --recursive

DeepStream-Yolo and librdkafka are used by the application as described in the introduction. First we need to configure and compile them.

Configure librdkafka:

$ cd 3pp/librdkafka
$ ./configure
$ cd ../../

Compile both third parties:

$ CUDA_VER=10.2 make subsystem

librdkafka needs installation:

$ sudo CUDA_VER=10.2 make install

Finally we compile the application:

$ CUDA_VER=10.2 make

Configuration

1. Models

The application uses a yolov4-tiny custom model for vehicle detection and mars-small128 model for DeepSORT tracking algorithm. Both models will be downloaded by running the following script:

$ bash models/get_models.sh

2. Kafka

Open the cfg/kafka_config.txt file and change the endpoint to reflect where your Kafka message bus is installed. The topic field may be left as it is.

If you don't already have a kafka message bus running you can check this simple deployment: zk-single-kafka-single.yml. You need to have docker and docker-compose installed on your machine.

Usage

The application takes as input a video in h264 format and outputs a mp4 video with the annotations:

$ ./bin/vehicle-tracking-deepstream test002.h264 output.mp4

For testing purposes you can download this video and use it as input for the app.

For tracking, the DeepStream discriminative correlation filter (DCF) is used but it can be changed to DeepSORT tracker by modifying the cfg/tracker_config.txt file. Just uncomment the ll-config-file line for DeepSORT and comment it for NvDCF tracker:

#ll-config-file=config_tracker_NvDCF_perf.yml
ll-config-file=config_tracker_DeepSORT.yml

Here is a video snippet with the NvDCF tracker (click on image to open the Youtube video):

IMAGE ALT TEXT HERE

And the same video but with DeepSORT tracker activated:

IMAGE ALT TEXT HERE

Discussion

The main issues for the trackers in this fixed camera scenario are the occlusions and the changes in angle of the vehicles. The NvDCF tracker suffers more from the occlusion problem. On the other hand the DeepSORT tracker handles the changes in vehicle angles worse. Nevertheless, in general, the two tracking algorithms perform similarly. At the end of the day almost the same number of vehicles are lost due to re-identification.

The DeepSORT algorithm uses mars-small128 model which originally was trained for recognizing humans. As a next step, it would be interesting to train a model for recognizing cars for DeepSORT tracker and check how much it would improve.

In terms of number of frames processed per second (FPS) there is no big difference between the two trackers. With DeepSORT the application processes around 14 FPS on average, whereas with NvDCF processes around 13 FPS.

About

Jetson Nano application that detects and tracks vehicles in a roundabout with Yolov4, DeepSort and NvDCF using DeepStream SDK and sends the info to a Kafka message bus.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published