How Uber is using driver selfies to enhance security, powered by Microsoft Cognitive Services

The second half of Uber’s formal name – Uber Technologies – isn’t always mentioned when people refer to the ridesharing giant. But technology is very much in Uber’s DNA, from its app that revolutionized individual transportation worldwide, to its current testing of self-driving cars in Pittsburgh.

“Technology goes through the lifeblood of the company, whether it’s figuring out how to make a process more efficient, or how to make the platform safer and more secure,” says Dima Kovalev, Uber product manager.

Now, Uber is introducing Real-Time ID Check, an additional security feature that periodically prompts drivers to share a selfie with Uber before they go online to start accepting ride requests.

Real-Time ID Check uses Microsoft Cognitive Services intelligence to instantly compare the selfie to the photo corresponding with the driver’s photo on file. If the two photos don’t match, the driver’s account can be temporarily deactivated while Uber looks into the situation.

This feature prevents fraud and protects drivers’ accounts from being compromised. It also protects riders by building in another layer of accountability to the Uber app to let passengers know that the right person is behind the wheel.

The verification feature is an example “of a great use of technology to reinforce the security foundation of our platform,” Kovalev says.

Real-Time ID Check is also incredibly fast – with the verification done in milliseconds.

“We have tens of thousands of driver-partners going online every hour across the world, and for us, the API’s response times were extremely important,” says Kovalev.

This feature builds an additional layer of security onto what’s already built into the app. Once passengers request a ride via Uber’s mobile app, Uber sends the rider information about the driver coming to pick them up, including the driver’s name, photo, car make and model, as well as the car’s license plate number. When the car arrives, riders can compare this information with the vehicle and driver they see before them to ensure they’re getting into their Uber.

Real-Time ID Check is not only adept at matching photos, it’s smart enough to detect anomalies or changes in a driver’s appearance, such as whether a driver now sports a beard, even though his official Uber photo shows him without one.

“If a driver’s photo doesn’t match, Real-Time ID Check recognizes that it may be because they’re wearing glasses or a hat” in their selfie that day, says Joe Sullivan, Uber’s chief security officer. “So it will ask them to remove their sunglasses and retake the photo.”

Over the last few months, Uber did a test run of Real-Time ID Check with more than 50,000 drivers in various cities across the U.S., where the feature will be rolled out first.

Among the things Uber learned from testing: “Taking a selfie at night is really, really hard,” says Kovalev. “So we rebuilt our entire user interface (UI) so that it’s a very bright UI,” almost acting like a front-facing flash, “making it easier for partners to verify when they’re going online at night.”

And it’s important that drivers stay safe while taking their photos. That’s why the driver selfie must be verified before he or she can access the platform, preventing any picture taking while driving.

Microsoft Cognitive Services was “very responsive” to Uber’s needs, Kovalev says. The original API supported 1,000 users, “and we basically asked, ‘Hey, can we make that a million?’” and the Microsoft team working on the project “didn’t blink,” he said.

“The other part that was helpful through the API is that there wasn’t a standard requirement for the type of photo” to be sent, with drivers sending photos from cellphone cameras with different screen sizes, resolution and aspect ratios.

“There wasn’t a strict requirement to say that it had to be a certain size – there was a minimum, but it was fairly easy to meet that minimum,” Kovalev says. “It wasn’t like, ‘Hey, you have to have this kind of photo to compare only against this type of photo.’ And it didn’t really matter whether it was Android or iOS, or whatever capture device they were using to compare that photo against,” which made the API development faster, with that flexibility being “something we were definitely looking for.”

Andrew Shuman, corporate vice president of products for the Technology and Research team at Microsoft, says the API used for Real-Time ID Check is one of Microsoft Cognitive Services’ REST APIs, designed to give developers the ability to tap into high-quality technologies with just a few lines of code.

“We have expressly developed our APIs to be easy to use, regardless of which platform or language the developer chooses,” Shuman says. “The algorithms are high quality, tested and based on 25 years of Microsoft Research.”

Says Sullivan of Uber: “Excellent security is frequently almost invisible,” And Real-Time ID Check is an example of that, he says.

For more information on Microsoft Cognitive Services, visit Microsoft.com/cognitive.

Top photo: Uber’s latest security feature, Real-Time ID Check, uses Microsoft Cognitive Services to protect both riders and drivers.