Heavy vehicle regulator taps into data lake and keeps Australia truckin’

There is a fleet of trucks keeping Australia’s economy moving each day, ferrying goods and produce the length and breadth of the nation. Road transport contributes $137 billion to the economy. Australia’s freight task is project to increase 86% by 2031 and is growing faster than Australia’s population growth.   

 It’s the job of the National Heavy Vehicle Regulator (NHVR) to make sure they are productive – and critically, that they are operating safely. 

 The NHVR is Australia’s independent statutory regulator for all vehicles weighing more than 4.5 tonnes – and there are around 930,000 truck registrations to keep track of.  

 Established in 2014, as a national body, the NHVR operates across all states and territories except Western Australia and the Northern Territory.  

 As part of that transition it has established a Safety and Compliance Regulatory Platform – essentially a repository for all the data sets about the national fleet that it and its partners can then tap into for insights about how best to monitor and manage the trucks on our roads. 

 With the Azure based data collection growing steadily and Power BI providing the analytics to make sense of that data, the Regulator also has its sights on using artificial intelligence and telematics to collect and interpret a rich array of data and identify patterns and trends that will keep drivers and roads safer than ever. 

 Data sets already loaded included the registration details of about 930,000 heavy vehicles, accreditation customers, more than 60,000 access consent requests a year and a wide range of compliance data from transport and police agencies.  

 NHVR Safety & Compliance Regulatory Platform program relations and product manager Tammy Wigg explains: “The National Heavy Vehicle Regulator wants to be able to use evidence and insights to be an intelligence led, risk–based regulator. We want to be able to optimise what we do to target the highest safety risks first and foremost. Instead of intercepting 100 trucks at random, we want to use information and insights to be more selective in which trucks we intercept. This means compliant operators and drivers will be intercepted less frequently and can get on with earning a dollar. At the end of the day, we want a highly productive and very safe heavy vehicle industry.”  

Safe, productive, sustainable 

 There are three key pillars that the NHVR is focussed on promoting: safety, productivity and sustainability – with the primary focus on safety. 

 To support that it has been prioritising the collection of safety data sets which starts with registration data sets, but has added in a range of other information including heavy vehicle defect data and crash data. 

 “There are five risk profiles that we’re trying to acquire data for to help us optimise what we do. The first risk profile is to do with operators. What’s their appetite for compliance, noncompliance? What’s their history of intercepts, infringements? What’s their involvement in crashes?” says Wigg.  

 “The second profile is the vehicle profile. What’s the life of the vehicle? When was it first bought? When was it sold? When was it written off? When was it stolen? Maybe it got re-birthed? How are the various makes and models performing over their life? Are some makes and models performing really well when the vehicle’s over 30 years old, while other vehicles are performing less effectively? The vehicle’s maintenance history can also determine when it really comes to end of life. Even down to a very specific VIN number, how is that particular vehicle performing?” 

The NHVR also analyses information about where vehicles are going and where they are restricted to go – insights that can help inform policy making and enable the NHVR to advocate for infrastructure investment on behalf of industry. Supply chain parties who are using heavy vehicles to transport goods are also monitored to ensure that operators and drivers aren’t being nudged toward risky behaviour by the unreasonable demands of clients.   

It’s a lot of insight – and that demands a lot of data which has been sourced from multiple jurisdictions, different agencies and external sources – all of which use different computer systems and data formats. The NHVR’s approach has been to corral that information in the cloud where it can be accessed and used to greatest effect. 

The data and the detail 

The NHVR worked with Microsoft partners HCL and Dialog to scope its needs and identify the most appropriate platform that could centralise the NHVR data and both provide a store and source of insight for NHVR itself – but also an asset for NHVR’s own customers who want to access data sets via the NHVR portal. 

The NHVR portal enables operators of heavy vehicles to apply for permits and view their heavy vehicle registration information. The NHVR portal is expanding and will soon become a one stop shop for all NHVR services.  

 The Safety and Compliance Platform was architected for the Azure environment between partners and internal experts to meet the organisations requirements. All the NHVR systems now integrate with the Safety and Compliance Regulatory Platform, enabling important information to be accessible by NHVR staff and operators any time of the day or night.    

Already a Microsoft user, the NHVR is rolling out Dynamics 365, Office 365 and SharePoint online across the enterprise. Adding Microsoft Azure into the mix as the platform for the data lake, and using Power BI for analytics was deemed the best and most trusted approach. Data supports the NHVR’s strategic plan to be a risk based regulator. 

To reduce the barriers to data sharing the NHVR has taken an “accept all” approach to data, whatever format it comes in. 

Instead of requiring agencies to supply compatible data formats Wigg says: “Aging legacy systems are genuine constraints for our data suppliers. Rather than ask for harmonised or perfect data, we say ‘We will take your data as is.’ 

  “Even if a vehicle defect is described differently between Queensland and New South Wales, it’s still helpful for an on road officer in South Australia to know that a minor defect was issued for an intercepted truck two days ago in Queensland. The officer won’t issue yet another defect given that one has already been recently issued. This is a much better outcome for operators.  

 “Conversely, if there is a major defect still open for the vehicle and it hasn’t been rectified in the allotted time, the officer may be required to take additional action,” says Wigg. 

 Data provided to the NHVR is initially stored in Azure blob storage and then transitioned to the Azure data lake. In July 2018, 4.5 million registration records were available in the data lake. In a year with the additional data acquired, this has grown more than 70 fold to 336 million records. Additional mobile automatic number plate recognition technology is planned for the coming year which is expected to see the data growth at 50 fold increase year on year. 

 “We do have quite a sophisticated architecture so that we can take data in all kinds of formats and manage it effectively and from all sorts of sources,” Wigg acknowledges.  

 Azure also supports the NHVR’s requirement that the data lake is highly scalable as it will ultimately hold more and more big data, accepting information second-by-second from sensors such as Weigh–in–Motion detectors and truck sightings made on the roads. 

 “We know that we will need to scale and we want to be able to do that quickly and painlessly. But we don’t know for sure how fast or when. We can scale it up and pay more if we need to and scale back if we don’t need to, so we aren’t paying for a service that we don’t’ need,” says Wigg. 

 The NHVR Safety and Compliance Officers often work in some very remote locations with mobile applications that enable them to access critical information in real time, such as vehicle registration. The applications also allow them to record information about intercepts, defects and infringements electronically, and this information is available in real time to others in the field or back in the office.   

To ensure that NHVR employees can make evidence based decisions in an agile, efficient way, the NHVR is promoting self-service analytics using Power BI, and has trained around 90 of its staff.  

Wigg adds: “It is important that we train our people to understand the quality and completeness of each data set and interpret the data with appropriate care.” 

The self-service training is delivered at two levels. The first level, called “Consumer” training teaches employees to use already developed Power BI reports and dashboards and how to use the quality and completeness reports for data sets that underpin those reports.  

The second level is called “Creator” training. The training consists of a full day of training so that participants can use Power BI Desktop to create their own reports and dashboards to answer simple questions. This is followed up with an “embedding” program where participants are trained in the data model, basic business analysis and data analysis skills, and supported report development over a period of six months. 

“The intention is for our business users to be able to answer simple questions quickly and effectively for themselves. This will free up our analytics unit to focus on complex data analysis that requires highly sophisticated skills,” explains Wigg. 

Trainees participate in a Power BI community to share reports they have built and reduce duplication and development effort. The community will peer review each other’s work to ensure a high level of quality and reduce the risk of “doing dumb things with data”. The community also provides “crowd support” and participants can share knowledge and support other community members.   

Adding AI for insight 

Wigg expects even more insights will come from the use of artificial and machine learning in the future.  

“The reason we’re looking at AI and machine learning is because we are planning to acquire a large amount of data and, some of it would be coming through in pretty regular cycles.  

“We can’t manually adjudicate this information and we’re not going to get 200 more resources to sit there and analyse data either. We’re looking for features in Microsoft that will help us again, optimise our understanding of risk and to be able to focus our limited resourcing and time on where the highest risk is, and continuously monitor what we think is our risk, because that risk profile could change. 

“There’ll be patterns that we are looking for that we want to be able to define in there. But then there are also patterns we were never looking for that we are hoping AI will help alert us to as well.“ 

The strategic goal is to maximise the impact of data for the NHVR that will in turn boost safety, promote productivity and encourage more sustainable practices that will ultimately have a national economic impact.