Autonomous tracking of vehicle taillights from a mobile platform using an embedded smart camera

The detection and tracking of vehicle taillights is an important aspect of collision avoidance systems and autonomous vehicle applications. In this paper, we present a novel and efficient algorithm for tracking the vehicle taillights from a mobile platform, under both daytime and nighttime conditions, which is entirely implemented on a CITRIC embedded smart camera. The algorithm uses a Kalman filter and a codebook to achieve a high level of robustness. On the microprocessor of the camera, it takes about 177 ms to process one frame of live camera data (which translates to approximately 6 fps). We demonstrate lightweight and reliable tracking of vehicle taillights, despite foreign objects appearing in view, blocking the view, or the vehicle changing lanes. In all of these cases, the algorithm is able to gracefully recover and resume normal operation. We will use this system as an initial platform for implementing other functionality, such as the detection of vehicle alert signals (brakes, turn signals, emergency flashers), which is also discussed. Compared to most existing work that focuses only on nighttime detection, the proposed algorithm provides daytime tracking of taillights, which is inherently more challenging.