NSW Police steals a march on criminal activity with cloud and cognitive capabilities

NSW Police Force pulls in about a terabyte’s worth of computer vision each and every day to support investigations – including from CCTV, from body worn cameras, laptops, mobile devices to dashcams.

Add into that mix recordings from interviews, information from triple zero calls, police reports, forensic investigations and it is clear how the data collection available to support enquiries expands exponentially.

This could be needle in a haystack territory. But it’s not thanks to a massive digital modernisation program well underway that brings to bear cloud computing, artificial intelligence and machine learning, edge computing and mobile devices to ensure that front line officers have the information they need to expedite justice and are not deskbound and data mired.

NSW has the largest police force in Australia with more than 22,000 members, including 18,000+ police officers serving 8 million people. Since 2017/18 NSW Police under a new Digital IT Strategy and leadership has digitised much of the interactions and channels it has to NSW citizens – from supporting them to  report crime across all types to doing a national police check in seconds online. Internally, through a platform NSW Police call BluePortal officers can request everything digitally from a search dog, PolAir, mounted police to radios and IT equipment as well do a risk assessment and operational orders or just enquire on their roster or payroll issue.

As part of this significant digital transformation effort, which has been steered by Gordon Dunsford, CITO and Executive Director Digital Technology and Innovation, more than 200 of NSW Police’s legacy systems are being retired, re-architected or replaced with modern cloud-based systems.

One of the landmark programs of work is the Integrated Policing Operating System (IPOS). This modern cloud-based platform replaces NSW Police’s 27-year-old central database and will be used to manage all the data from operations including triple zero calls, arrests and charges, firearms, criminal investigations and forensics – and then make that easily accessible by police officers.

IPOS is based on Mark43’s public safety software and NSW Police is working with Microsoft Consulting Services on the build of the IPOS application with Protected level security in the Microsoft Azure cloud.

At the same time NSW Police has rolled out its AI/ML-infused Insights policing platform which speeds up significantly, and democratises, access to a wide array of critical information and automates many tasks such as transcribing recordings of audio interviews or poring through petabytes of CCTV footage. This ensures faster access to useful insights and frees police to focus on operational matters.

The text-to-speech capability of the platform is already attracting interest from other agencies internationally, at Federal and State levels, says Dunsford.

It’s also dramatically speeding up police work. A 20-minute recording of a statement can take an individual Police officer two or three hours to manually transcribe. With the Insights platform this is now completed in seconds or minutes with a good-to-great level of accuracy in the automatic transcription –which is only getting better thanks to machine learning.

The system has been designed with ethics front and centre, and in consultation with privacy experts with a particular focus on avoiding bias.

Since establishing its, Office of Responsible AI, Microsoft has undertaken reviews of many AI/ML solutions. These reviews are conducted by the Office of Responsible AI, with support from a globally diverse group of individuals, and examine key aspects of AI-based solutions,  through the lens of our responsible AI principles and considering potential impact to human rights.

In June 2020 Microsoft confirmed that it would not sell or deploy facial recognition to United States police services. This solution aligns with that commitment because it does not use facial recognition. It uses the Azure computer vision API to identify objects that assist with police cases, bringing significant value to the NSW police in their pursuit of justice.

NSW Police’s AI/ ML infused Insights platform has been tried and tested to great effect on latent data. This reflects all the types of data collected for the purposes of helping to speed up investigation processes – more than halving the time officers need to spend wrangling data manually or just assessing video footage.

In one case, NSW Police collected 14,000 pieces of CCTV as part of a murder and assault investigation that would previously have taken detectives months to analyse. The AI/ML infused Insights platform ingested all the CCTV for the analysis in around five hours. Detectives were able to then within days piece together the time sequence of events, movements and interactions of the person of interest as well as overlay this onto a geospatial platform, visualising the data for detectives and aiding in preparation of the brief of evidence for Courts.

IPOS and Insights are two important building blocks of the Digital Evidence Cloud that Dunsford and his team are building to transform policing in the State and accelerate access to justice for victims and families who are affected by crime.

Accelerated investigations

Traditionally, front line officers have often had to spend hours transcribing recordings of interviews or reviewing video footage, taking them away from other duties.

The Insights platform can turn voice to text and also stitch together CCTV with dash cam footage and then search for objects of almost any type, including overlaying this on a geospatial solution for complete data visualisation.

Dunsford explains: “Using computer vision it can search to recognise objects, vehicles, locations, such as a backpack, or a tie, or type of shoes a person of interest might be wearing.

While IPOS and Insights are independent solutions, they are very much complementary.

IPOS brings together information from triple zero calls, complaints or reports from the public, all the way through to data from investigations management, forensics and ultimately custody management.  IPOS brings to life the ‘steel thread’ processes of NSW Police Force from the call for service through 000 to the hand off to the Justice or Courts systems.

The IPOS system gives a single view of a person of interest across all the functions of a large and complex law enforcement Agency like NSW Police and can make that information available directly to police via their MobiPol mobile devices. It can also provide access to important additional information – for example alerting police to the fact that the address where they are going to apprehend someone is located next to another house where residents are known to be antagonistic to the police through its geofencing capability.

Dunsford says that at present NSW police officers share MobiPols – but with IPOS there are plans to equip every officer with their own device and access to IPOS.

Joseph Ontedhu, the NSW Police Chief Enterprise Architect, who has played a key role in the development of the Insights platform meanwhile describes it as an investigation support tool that can accelerate the pace at which useful information can be provided to police.

Insights is currently hosted internally, but shortly will migrate to the cloud, and already NSW Police is using a containerisation strategy to parcel up data that needs to be interpreted rapidly, and sending that to Azure for processing.

Dunsford says Microsoft Azure’s security credentials are highly valued by NSW Police, helping it to de-risk its modernisation program. The selection of Azure as the platform for IPOS was also supported by Mark43’s decision to build its software on the Microsoft cloud.

NSW Police is supported by Microsoft Consulting Services as it builds IPOS to Protected level in Azure.

Dunsford adds that the performance and scalability of Azure supports the bursty nature of some police work, particularly State Crime and State intelligence Command work where there can be huge volumes of information to process and manage in a short space of time.

Azure also plays a key role in the democratisation of data – making information available to police where and when they need it, preventing them being deskbound.

Ontedhu explains that if the information that an officer needs is sitting in a legacy system, the metadata about that information can be brought into Insights for analysis. “Let’s say, they want to know if NSW Police has got any CCTV or body-worn video that NSW Police now wear, or even from a police car specific to a certain location at a certain time, they can go search. Then we’ll get a hit if that body worn video has been uploaded.

NSW Police already has plans to replace the legacy data store systems with the Digital Evidence Cloud, and has built a small-scale capability and has trialed this with NSW Police’s Forensics Command.

That, says Dunsford, should dramatically increase the amount of material that can be collected and analysed to support current and future investigations when NSW Police is ready to scale the trial. “We join the dots,” he says.

Insights has already proven its ability to join the dots by supporting Strike Force Toronto which looked for evidence of arson and other criminal acts from the 2019/20 NSW bushfires.

Dunsford said that during investigations about the 2019/20 bushfires the system took geographic information, bushfire information, and then overlaid that with data from some 17 separate sources including Bureau of Meteorology data about lightning strikes and wind conditions. It then overlaid that with geo-located mobile phone data of suspected arsonists.

That, he said, had allowed police to present evidence in a very graphical or visual way to the Bushfire and related inquests investigating the horrific bushfire season in NSW.

Dunsford says the impact Insights is having means demand for the platform has taken off; “Because of the speed, the agility and the ease of use in particular. A lot of this comes from the ability to scale, the ability to throw GPUs at scale at a platform like this, and harness the peaking capability we need to run these major, major investigations with tens of thousands of hours at CCTV and other digital evidence.

“That’s what the cloud offers NSW Police, and particularly Microsoft.”

Building tech intensity

Dunsford has made significant progress already on building strong digital foundations for NSW Police Force and has also carved out a reputation for the organisation as being a leading-edge innovator that attracts and retains leading edge digital tech talent.

It’s this combination that delivers real tech intensity to an organisation. According to Dunsford; “I want us to be seen as an organisation that’s very much investing in technology and talent to really transform what we do with frontline police and the way they work. How they use technology to get results faster, smarter, not having to work harder and longer, not burning out.

“And being out in the street more, so everything they do in a police station, they can do on the street from a mobile first perspective.”

With the Digital Evidence Cloud now well advanced and ready to be scaled off the back of IPOS and Insights, he is exploring the potential of further innovation with a planned trial of Microsoft’s edge technology in NSW police cars.

Dunsford also wants to understand how low earth orbit (LEO) satellites could be used to support police; how data from the Integrated Connected Officer program which collects data from an officer’s firearm, taser, car and body worn camera can be ingested into Insights; and how drones could be deployed to collect video that could help identify  potentially dangerous situations.

Advanced AI and machine learning could, he thinks, be used to train systems to identify everything from the colour, make and model of vehicles, to a backpack in a crowded street, to finding a particular individual based on their unique gait.

Dunsford stresses that as AI/ ML plays an increased role, ethics considerations are at the heart of all the NSW Police’s modernisation plans and that it will always consult stakeholders. The goal however remains crystal clear – to digitally enable and keep police safer, to prevent and disrupt crime and to speed up justice outcomes for victims of crime and the community.