Black.ai turns fiction into fact with all-seeing cloud solution

From Harry Potter’s Marauder’s Map to Q’s ability to follow James Bond’s movements through an abandoned WWII submarine base, fiction has long celebrated smart tools that allow us to see more than the naked eye.

Melbourne-based software business Black.ai is turning fiction into fact with a platform that can map what’s happening pretty much anywhere in a physical environment using data from CCTVs and movement sensors.

Established in 2016, the company is now deploying its platform internationally in retail, health, heavy industry, and oil and gas sector settings.

Four years of deep R&D were required to get to this stage says co-founder and CEO Keaton Okkonen, who explains that the challenge has been to find a way to create “stateful understanding of a space.”

Okkonen says; “In the software world or in a computer, you’ve got information that is going to represent the state of a system in pretty predictable channels and formats, because we build computers and it’s really easy to operate within that sort of space. As a result we have efficient software systems. We can optimise them easily.

“But within physical environments, we don’t have access to that same sort of stateful understanding of the space. And it’s actually really hard to get that. And then by extension, it’s hard to build  efficient physical systems.

To understand that gap, consider a single CCTV image – viewed in isolation it can be considered “stateless” – interesting, possibly, but not very useful.

In order to establish “stateful” understanding of a physical space Black.ai uses machine vision and AI to provide context and a more comprehensive understanding of what is going on, combining CCTV images and sensor data and then interpreting what is going on.

The platform can also be used to control what happens in the physical environment if something is observed.

For example, Black.ai can notify when someone is working by themselves in an area where safety procedures dictate they should work in pairs, or when a hazard is left on the ground in a thoroughfare area.

This real time digital twin of a physical environment generated by Black.ai provides clarity about what is going on in a particular location, and the opportunity to take action either manually or automatically.

Black.ai’s platform also allows the physical environment to be searched.

Okkonen says; “We built out a system that you can put into any space and then we can run off either CCTV or sometimes specialised depth cameras that will map out that environment and understand what’s going on where. So, where are the people? What are the people doing?

“Then you can program against changes to the state of that space.”  This is where things get really interesting.

Cross sector impact

Black.ai’s platform can capture and analyse 30 frames per second from almost any number of cameras installed in a physical space. It can do that either in real time, or historically.

It has been tested in environments from 400 square feet to 40,000 square feet, and can monitor the behaviour of hundreds of concurrent visitors to the location. Black.ai’s platform has deliberately avoided using facial recognition technology, and anonymises any content that could identify an individual in order to protect people’s privacy.

Black.ai is currently working with an Australian resources giant to explore how the system can be used to automate parts inspection, with expectations that it might reduce the cost of quality assurance inspections by as much as 30%.

It’s also being tested to help automate minerals grading and inspection. Black.ai says the system can also be used to monitor workers’ proper use of PPE (personal protective equipment) to reduce risk.

In retail Black.ai’s vision infrastructure has a variety of applications says Okkonen with trials underway with a number of major retailers in Australia and the US.

While not conceived as an anti-theft solution, it has been applied to programmatically control for loss by suggesting bag checks when there is a large disconnect between items a shopper has selected on shelf vs paid for at checkout.

For supermarkets battling meat theft the impact can be significant, he says.

“The retailer is on a 2-3% margin, meat makes up 20% of their revenue, and we found that 5-8% of all meat items taken from the shop are stolen. If you can programmatically remove that, the retailer can effectively double their bottom line.”

Health applications include monitoring patient vital signs, patient movement, and any visits from clinicians or family.

Multiple industries could benefit from real time tracking of people and assets according to the company.

Designed specifically to make use of existing assets such as CCTV cameras and sensors, Black.ai has been very deliberately developed on Microsoft Azure.

“Beyond CCTV, we also use the Kinect sensors quite a lot – they are the best, cheapest depth sensor on the market.”

Okkonen says that the Microsoft Azure foundations also streamline Black.ai’s ability to grow and scale internationally.