How AI, drones and cameras are keeping our roads and bridges safe

“It's a dangerous business, Frodo, going out your door. You step onto the road, and if you don't keep your feet, 

there's no knowing where you might be swept off to.”

― J.R.R. Tolkien, The Lord of the Rings

Europe’s roads are the safest in the world. Current figures show that there are 50 fatalities per one million inhabitants, compared to the global figure of 174 deaths per million. Despite this, each loss remains a tragedy. In 2017, 25,300 people lost their lives on European roads.   

The cause of these accidents can vary from human error and weather conditions, to damaged structures and surfaces. While some things are beyond the realms of control, road and bridge conditions are a variable which can be governed.

As soon as a road is paved, a combination of traffic and weather conditions begin to degrade and erode the surface. Undetected cracks, abrasions or defects can quickly lead to bigger problems, such as costly repairs, major traffic delays, and in the worst cases, unsafe condition. These problems are also shared by bridges, particularly when concrete is critical in maintaining the integrity of the structure. The earlier faults are detected, the sooner they can be addressed, saving time and money, while minimising disruption. Ultimately, this helps ensure that the roads themselves are safer for those travelling on them.

The detection of these faults, however, can be very difficult to carry out manually, especially as early-forming cracks are hard to spot with the naked eye. Predicting where faults are likely to occur ahead of time so that appropriate measures can be taken in advance also possess a massive challenge. Thankfully, technology is here to help.

Building bridges
Built more than 20 years ago, the Great Belt Bridge is a suspension bridge which connects the Danish islands of Zealand and Funen. Holding company Sund & Bælt, which is responsible for the maintenance of the bridge, has worked with Microsoft to deploy an innovative solution which combines the flexibility of drones, with the power of artificial intelligence (AI.)

The drones are used to fly around the bridge and capture thousands of pictures of the concrete structure – a method that’s far safer and faster than tasking a worker to dangle 200 metres above the surface to take pictures manually. The expertise and experience of these workers is instead used to help train a machine learning algorithm which can automatically detect cracks in the surface of the concrete, after the photos have been uploaded to Microsoft’s Azure cloud. After the AI creates a list of areas with cause for concern, the same experts are used to select the areas which need maintenance and repair.

Left unattended, these cracks can grow large enough to expose the underlying steel frame of the bridge itself. If the steel then rusts, the strength is compromised, and rebuilding is the only option. “Concrete does not simply degrade overnight – it is a slow process. Therefore, being able to detect and predict potential points of damage in advance is extremely useful,” says Mikkel Hemmingsen, CEO of Sund & Bælt.

The end result in a process which increases safety while saving time and money. In addition, it has also allowed the company to carry its learnings over into newer bridge constructions. There are also plans to train the algorithm further, by applying the same method to the Little Belt Bridge, the Vejlefjord Bridge and the Øresund Bridge.

“Our core focus was to create a solution to maintain and improve efficiency, but we quickly realised that the more we used the solution, the better it actually became. This has given us an incentive to spread the solution to others,” Hemmingsen states.

From bridges to roads
Multinational construction company BAM Infra Nederland and OrangeNXT, a leading software integrator, have developed a system using Microsoft Azure, machine learning, and AI to train algorithms that can accurately detect and classify various types of damage on paved surfaces.

In the past, BAM sent out drivers in camera-equipped cars to capture photos and videos of road surfaces. Inspectors then reviewed the content to identify damaged areas, before marking them and making a plan to fix them. “This process was time-consuming, costly, and tedious,” says Kitting Lee, Director of Commerce and Innovation at BAM Infra Nederland. “We needed a smarter solution.”

The new solution sees vehicles equipped with 360-degree cameras record video footage from every angle, before they are uploaded to the Azure cloud, where AI-powered algorithms automatically flag any causes for concern. These images also capture geospatial data, allowing inspectors to accurately trace them back to their real-world location. This improves the speed, quality, efficiency, and accuracy of these visual road checks, enabling predictive asphalt maintenance, while reducing costs – freeing inspectors to focus their expertise where it is really needed.

“Most roads,” Lee continues “were being checked just once per year. We knew that if we could check more frequently, we could prevent small defects from becoming big holes, which would improve public safety, enable predictive maintenance, and reduce emergency repairs that shut down roads and cause traffic jams.”

Beyond the obvious efficiency increases in time and cost, the new system also makes for happier employees, while attracting new talent. Rather than trawling through hours and hours of footage from undamaged roads, inspectors can now focus purely on sections which require attention, leading to quicker repairs, and higher job satisfaction.

Given the success of the new asphalt-inspection solution, OrangeNXT and BAM are also considering selling the new process as a software-as-a-service solution in other countries and for other purposes, opening up a new business opportunity while ensuring that they remain competitive.

From bridges and roads, to everything in between, technology such as AI is helping to ensure that the paths we travel down are as safe as they can be.