At any given moment, turnaround coordinators for German airline Lufthansa CityLine have their eyes glued to monitors displaying more than half a dozen video feeds of airplanes parked at gates around the airport. The coordinators’ job is to ensure that the planes are unloaded, refueled, cleaned, restocked and reloaded so that every passenger reaches their destination safely, on time and with their luggage.
Minutes lost here or there in the turnaround process can add up, costing airlines millions of dollars a year. As many in the industry note, airplanes only make money while in the air.
“Think of a pitstop in a car race, and this is pretty much the same that happens in a turnaround for an aircraft,” said Philipp Grindemann, head of business development and project management for Lufthansa CityLine. “All the processes need to be on time, need to be fast, need to be lean.”
Lufthansa CityLine is a subsidiary of Lufthansa, one of the world’s major airline groups with a network that spans the globe. Lufthansa maintains hubs in Frankfurt and Munich, Germany. Lufthansa CityLine connects passengers with destinations around Europe to and from these hubs, flying more than 300 flights per day. Ontime arrivals and departures are essential for customer satisfaction and Lufthansa’s bottom line.
Outside of weather, delays stem from missteps during the tightly choreographed turnaround process. Like most industry players, Lufthansa CityLine relies on manual timestamps to understand when each step of the turnaround process starts and ends and uses that manual timestamp data to glean insights on where to make adjustments for faster, leaner turnarounds.
A ground operations crew for Lufthansa CityLine positions an aircraft during the turnaround process. The airline is piloting AI technology that can help improve the efficiency of the turnaround process. Photo courtesy of Lufthansa.
In a pilot phase, the airline partnered with zeroG, a Lufthansa Group consulting company founded by Lufthansa Systems to accelerate the tangible impact of artificial intelligence in operational and commercial processes at airlines around the world. One example is improving turnaround management with AI.
ZeroG’s Deep Turnaround solution leverages Azure Video Analyzer, a new offering from Microsoft that combines capabilities from Live Video Analytics and Azure Video Indexer. For Lufthansa, it generates automatic timestamps from the video feeds and issues alerts when the turnaround goes off script.
“With that transparency of Deep Turnaround – knowing when the caterer arrives, knowing when the bridge arrives in order to deboard the aircraft – the airline can steer the process and have much leaner processes than before,” said Manuel van Esch, lead consultant for zeroG.
For example, when a fuel truck arrives later than predicted, Deep Turnaround alerts the turnaround coordinators and other ground operations personnel. The alert kicks off a hunt for a solution that prevents a delay, such as dispatching a second fuel truck to the plane.
Applied AI Services
Azure Video Analyzer is among a handful of Azure Applied AI Services that Microsoft highlighted on Tuesday during Build, the company’s annual conference for developers. These services – Azure Video Analyzer, Azure Metrics Advisor, Azure Bot Service, Azure Cognitive Search, Azure Form Recognizer and Azure Immersive Reader – accelerate the development of scenario-specific AI solutions.
Azure Applied AI Services are built on top of AI models at the core of Azure AI products and services. That includes Azure Cognitive Services, which offer customizable AI models and tools for building AI solutions that help customers extract meaning from text, integrate speech into apps and services, identify and analyze content within images and videos, and make decisions.
Customers can also customize these services and extend them with their own custom models from Azure Machine Learning to meet the specific needs of their business.
Customers routinely tell Microsoft that while they see the potential of AI, building solutions are harder than they anticipated, said Eric Boyd, corporate vice president of Microsoft Azure AI in Redmond, Washington.
“The goal with Azure Applied AI Services is to provide a bit more packaging and structure to really accelerate the development of AI solutions for common business processes,” he said.
The Azure Video Analyzer service, for example, brings together Computer Vision from Azure Cognitive Services and an automatic captioning model along with capabilities for integrating existing closed circuit video feeds and video management systems, which make it easier for businesses to build video analytics solutions.
Microsoft created the Azure Applied AI Services category to target common business scenarios that Boyd’s Azure AI team has seen customers repeatedly build from scratch. For example, Azure Form Recognizer builds on optical character recognition, a computer vision technology that recognizes text and is key to many business solutions from reading receipts to pulling data from intake forms.
“To put it in the application that they wanted, there was so much more they needed to do,” Boyd said. “It wasn’t just about getting the text, it’s about understanding the structure of the document and saying, ‘I have this form that someone has filled out, and I want the information on it in my database.’”
Azure Form Recognizer builds on the underlying optical character recognition technology with a framework to understand the whole document structure, extract the relevant information and populate a database.
Building on Microsoft’s internal AI solutions
Many of the Azure Applied AI Services build on AI tools originally developed for internal products and services, including Azure Metrics Advisor. The tool grew out of work developers did for Microsoft’s search engine Bing to detect deviations from normal operations, such as spikes in queries from one country or a sudden drop in advertising revenue.
“Search is really quite predictable in how it changes day over day, and so being able to detect those anomalies, we could really more quickly jump on the problems and solve them,” Boyd said. “That anomaly detector service has rolled out to a number of places, such as Power BI. But it’s a developers’ interface and requires you to string a lot of it together.”
Microsoft made the technology available to the public through Anomaly Detector, one of the Azure Cognitive Services. For Azure Applied AI Services, Microsoft built on the technology that powers Anomaly Detector and tailored it to common solutions for business customers, making it easy to deploy a solution that monitors metrics and, when something is awry, issues an alert and flags where to look to resolve the issue.
Samsung Electronics deployed Azure Metrics Advisor in China for anomaly detection and root cause analysis of problems that could lead to outages on the cloud-based hardware and software system that enables around-the-clock access to audio and video content streamed over the internet for display on the company’s Smart TVs.
The AI solution built with Azure Metrics Advisor helps Samsung engineers detect incidents before they affect customers and quickly fix the issues, said Jie Zhang, the technical leader of Samsung Electronics (China) R&D Centre, who helped on design and implementation.
The backend development of Azure Bot Service followed a similar trajectory as Azure Metrics Advisor, Boyd noted. That service builds on the core speech and language technologies that power Azure Cognitive Services such as Language Understanding, QnA Maker, Speech to Text and Text to Speech in order to help customers develop intelligent conversational assistants.
“We combine a number of Cognitive Services together and package it up and make it simpler for users to use the whole combined thing,” Boyd said.
More value with AI
Several of the services now available under the Azure Applied AI Services category were previously available as standalone Azure Cognitive Services such as Azure Form Recognizer and Azure Immersive Reader, which enables developers to implement techniques into their applications that improve reading and writing for people regardless of their age or ability.
Other services were individual offerings under Azure AI, such as Azure Bot Services and Azure Cognitive Search, which allows developers to integrate AI-powered search to their apps and websites.
The reorganization, Boyd said, is intended to make it easier for business customers to find AI solutions to common business processes. The new category is expected to grow in the months and years to come as the Azure AI team works with customers within specific industries or sees customers solving the same problem with a combination of Azure AI services.
“We see this category of ‘How do you package and combine and simplify these things?’ really speaking to people understanding that there’s tremendous power in what AI can do and saying, ‘I need to bring that to all areas of my business,’” Boyd said.
The ability of AI to generate and make sense of data is behind zeroG’s embrace of Azure Video Analyzer for the Deep Turnaround solution piloted by Lufthansa CityLine. And this is just the beginning of a digital transformation with AI throughout the Lufthansa Group, according to Xavier Lagardere, the aviation company’s chief data officer.
“We’re not yet systemically a real time data-driven type of company when it comes to making choices, making decisions or even acting upon data,” he said. “There’s an exciting road ahead to do much more with the huge piles of data that we generate in the day-to-day.”
Related:
- Learn more about Azure Applied AI Services
- Read: What’s that? Microsoft’s latest breakthrough, now in Azure AI, describes images as well as people do
- Read: Microsoft responsible machine learning capabilities build trust in AI systems, developers say
- Read: Microsoft services helps healthcare organizations develop and deploy virtual health assistants
Top image: A Lufthansa CityLine airplane is parked at an airport gate as ground operations crew prepare it for the next flight. The airline is piloting AI technology that can help improve the efficiency of the turnaround process. Photo courtesy of Lufthansa.
John Roach writes about Microsoft research and innovation. Follow him on Twitter.