Microsoft Research AI project automates driver’s license tests

What’s the first thought that comes to your mind when you think about the experience of getting a driver’s license? A typical scenario would entail you bringing your car to a test track and waiting to be assigned an evaluator. When your turn comes, the evaluator rides shotgun with you and evaluates your performance as you go through the manoeuvres. At the end of the test, the evaluator decides whether you passed or failed, based on their, often subjective, assessment of your performance. The result, whether you agree with it or not, is final.

What if we tell you that your driver’s license test will now be conducted by a smartphone attached to your car’s windshield and without an evaluator present in the vehicle? This is no science fiction. It is already happening at the Regional Transport Office (RTO) in Dehradun in the northern Indian state of Uttarakhand, courtesy of a Microsoft Research Project called HAMS.

HAMS, which is short for Harnessing AutoMobiles for Safety, was originally conceived as a system to monitor drivers and their driving, with the aim of improving road safety. Driver training and testing are foundational to this goal, and so the project naturally veered in the direction of helping evaluate drivers during their driving test.

Driver license testing is a pressing problem; for instance, a survey by SaveLIFE Foundation in India reports that a whopping 59% of the respondents did not give a test to obtain a driving license.

“The main challenge in the traditional driver’s license test is the burden placed on the human evaluators and the resulting subjectivity that a candidate faces. Automation using HAMS technology can not only help relieve evaluators of the burden but also make the process objective and transparent for candidates,” says Venkat Padmanabhan, Deputy Managing Director, Microsoft Research India, who started the HAMS project in 2016.

Padmanabhan developed HAMS along with Akshay Nambi, who joined the project as an intern and is now a Senior Researcher at Microsoft Research India, and Satish Sangameswaran, Principal Program Manager, Microsoft Research India, who helped the team bring HAMS out of the research lab and into the real world through external partnerships.

One such partnership with the Institute of Driving and Traffic Research (IDTR), a joint venture between the Department of Transport of State Governments and Maruti Suzuki India Limited, India’s largest passenger car manufacturer, resulted in the implementation of HAMS as a unique smartphone-based driving test system for issuing driver’s licenses.

HAMS, in its general incarnation, uses the smartphone’s front and rear cameras, and other sensors, to monitor the driver (for instance, their gaze) and the road scene in front (for instance, the distance to the vehicle in front), simultaneously. It employs advanced Artificial Intelligence (AI) models, which the team has developed for efficient and robust operation.

For driving tests, HAMS has been customized to include capabilities such as precise tracking of the vehicle’s trajectory during designated test manoeuvres, for instance, parallel parking or negotiating a roundabout. This tracking enables HAMS to determine precisely, for instance, whether the driver stopped in the middle of a manoeuvre for longer than is permitted or tried to course correct by rolling forward and backward alternately more times than allowed.

Today, if you take the driver’s license test at the Dehradun RTO, you will be doing so in just the company of a smartphone affixed to your car’s windshield. HAMS, running on the smartphone and on an edge server onsite at the testing track, will do the rest and produce a detailed report shortly after you finish navigating through the test manoeuvres.

“The successful deployment of the HAMS-based driver license testing at the Dehradun RTO is a significant step towards the Transport Department’s goal of providing efficient, world-leading services to the citizens of Uttarakhand. We are proud to be among the pioneers of the application of AI to enhance road safety,” said Shri Shailesh Bagauli, IAS, Secretary, Government of Uttarakhand.

“On an average, over 50 candidates take the HAMS-enabled automated license test every day at the Dehradun RTO. Due to the comprehensive nature of testing, just about 50 percent candidates pass the test, ensuring that only qualified drivers are given a driver’s license,” says Nambi.

While some RTOs in India are already leveraging automation by deploying extensive infrastructure such as pole-mounted video cameras along the entire length of the test track, HAMS not only helps bring down the cost of automation significantly by obviating the need for such infrastructure, it also improves test coverage by providing a view within the vehicle. For instance, it checks whether the driver scanned their mirrors before effecting a lane change, and even more rudimentary, whether the person taking the test is the same as the one who registered for it.

To ensure transparency, in case of a disagreement on the test results, applicants have the option of viewing a video recording of their test drive, to know if the system treated them fairly.

“HAMS stems from a line of work that I started, along with colleagues, when I returned to India from the US a decade ago and was aghast at the chaotic traffic on the roads. It is immensely satisfying for me that what we started as a research project is now helping make our roads safer,” Padmanabhan exclaims.

“At Microsoft Research India, we have been studying the role of technology in solving significant societal problems for over a decade now,” said Sriram Rajamani, Managing Director, Microsoft Research India. “The implementation of HAMS for driver license testing is a great example of the innovative use of AI to create transparent and efficient processes that ultimately help people, and improve road safety for all of us. I congratulate the Dehradun RTO and its administrators for being pioneers in this area.”

The success of HAMS at the Dehradun RTO augurs well for its wider adoption, within the country and beyond, to ensure well-tested drivers enable better and safer roads everywhere.