Return to Wisconsin Computer Vision Group Publications Page

Automatic driver face state estimation in challenging naturalistic driving videos
B. M. Smith, X. Wang, Y-H. Hu, C. R. Dyer, M. V. Chitturi and J. D. Lee, Transportation Research Board 95th Annual Meeting, 2016.

Abstract

Driver distraction represents a major safety problem in the United States. Naturalistic driving data, such as SHRP2 Naturalistic Driving Study (NDS) data, provide a new window into driver behavior that promises a deeper understanding than was previously possible. Unfortunately, the current practice of manual coding is infeasible for large datasets like SHRP2 NDS, which contains millions of hours of video. Computer vision algorithms have the potential to automatically code SHRP2 NDS videos. However, existing algorithms are brittle in the presence of challenges like low video quality, under- and over-exposure, driver occlusion, non-frontal faces, and unpredictable and significant illumination changes, which are all substantially present in SHRP2 NDS videos. This paper presents and evaluates algorithms developed to quantify high-level features pertinent to driver distraction and engagement in challenging videos like those in SHRP2 NDS. Specifically, a novel two-stage video analysis pipeline is presented for tracking head position and estimating head pose, and eye and mouth states. Results on challenging SHRP2 NDS videos are promising. The accuracy of the new head pose estimation module is competitive with the state of the art, and produces good qualitative results on SHRP2 NDS videos.