Multi-task faster R-CNN for nighttime pedestrian detection and distance estimation

Distance estimation and pedestrian detection are critical for safe driving operation decision-making and autonomous vehicle intelligent control strategies. This paper proposes a novel multi-task Faster R-CNN detector which simultaneously realizes distance estimation and pedestrian detection using an improved ResNet-50 ar-chitecture. Images were acquired using a near-infrared camera with two near-infrared fill-lights devices during real road nighttime scenarios. Ground truth pedestrian distances used for training were obtained using LIDAR. The data used to optimize the multi-task Faster R-CNN detector were approximately 20 k high-quality near-infrared images with marked pedestrians and tagged distance values. The proposed algorithm including the distance estimation runs at a speed exceeding 7 fps. Pedestrian detection accuracy reached nearly 80% with a total average absolute distance estimation error rate of less than 5%.