This project was one of the most complex ones. The ultimate goal was to detect driver fatigue and issue warnings to driver via alarms. In addition, this system should have been deployed into IOT device like Raspberry PI 4, which introduced some limitations in terms of algorithms and methods we can use.
Project Details
As usual, I have started with some research for best practices and found good papers which I have used for final solution. https://www.researchgate.net/publication/342193728_A_Fatigue_Driving_Detection_Algorithm_Based_on_Facial_Motion_Information_Entropy
The first step was to detect face in video stream using ONNX library and then detect facial landmarks for further analysis. The key point was to identify FFT and FFV (for more details please look into paper attached). Based on labeled data which was collected using simple web-camera we have trained an XGBoost model with current value and n-previous values of FFT and FFV (n=10). Based on it we were able to identify person’s head position also in time dimension approximately for last 3-5 seconds and based on that issue alerts if driver keeps head down and does not move it. Later means person is fatigued and cannot drive, whereas if driver actively looks around and watches traffic there should be no alarm issued.
In addition, if person was yawning or having eyes closed alarm has been also triggered by separate rule-based algorithms.
Eventually everything was dockerized and deployed onto Jetson Nano/ PI 4 IOT devices.