The goal of this project was to detect vehicles, their license plates, along with tracking already detected vehicles so that one vehicle would not be detected twice. Also, vehicle type and color classification was performed.

The data for this project was collected from three different datasets:

KITTI dataset  This image dataset includes over 14,000 images made up of 7,518 testing images and 7,481 training images with bounding boxes labels in a separate file.

MS-COCO dataset Images falling under the categories of car, bus, and van were manually downloaded along with their annotation information.

Stanford Cars Dataset This dataset includes 16,185 images with 196 different classes of cars.

The collected images were checked for quality and quantity among different classes to form a balanced dataset. Experiments were then performed using different distributions of this dataset on variations of the EfficientDet object detector for vehicle and license plate detection. For vehicle tracking, DeepSORT was used.

Classification of car type (make/model) and color were performed using mobilenet v2 with an accuracy of over 90%.

Overall pipeline from vehicle detection to tracking and classification was successfully run at around 20 frames per second on an Nvidia RTX 2060 GPU.