Coles deepens its relationship with shoppers using AI to understand the customer experience and improve efficiency in store
Microsoft News Center
Leading supermarket retailer Coles has been pioneering the use of AI technology with a visionary approach for nearly a decade. Its AI journey, which started with an ambition to transform operations, has matured into an impressive suite of AI models that now drive the day-to-day engine of its sprawling operations.
This AI-backed operational efficiency helps Coles predict the flow of 20,000 stock-keeping units (SKUs) to 850 stores nationwide with astonishing accuracy, harnessing insights from over 2,000 diverse data sets to make 1.6 billion predictions each day. Much more than mere numbers, these are the heartbeats of a system ensuring that every Coles customer finds exactly what they’re looking for, exactly when they need it.
AI is also helping Coles deliver more personalised shopping experiences. Over four million FlyBuys customers receive bespoke weekly product recommendations curated by an AI model that understands their preferences in the context of broader customer patterns.
In partnership with Microsoft, Coles is now shifting from initiating AI solutions to scaling them across its operations. A critical element here is democratising its data, empowering teams across Coles to leverage data, drive intelligent automation, and unlock value in unprecedented ways.
A smart and efficient shopping experience
Coles’ commitment to customer-centricity doesn’t see technological innovation as an end in itself. Rather, it’s a means to create more meaningful and efficient shopping experiences.
“Win-win solutions are those where we are helping our team members and our customers at the same time. Our technological investments into operational efficiency have translated into real, tangible benefits for our shoppers,” Silvio Giorgio, GM of Data & Intelligence at Coles Group said.
Coles has leveraged Microsoft Azure HCI alongside NVIDIA’s A16 GPUs to run large-scale AI workloads with great efficiency. One example of how this is benefitting the retailer and its customers is how Coles uses Computer Vision to revolutionise the checkout process, particularly in the selection and scanning of fresh produce. The traditionally manual task of scrolling through multiple options to identify and select produce has been simplified. Computer Vision systems can accurately scan items – even through plastic bags – identifying them with a high level of accuracy. This streamlining substantially reduces wait times and enhances overall customer satisfaction at checkout.
Coles has also identified an opportunity to improve queue monitoring at its supermarket deli counters. The company found that customers are likely to abandon their purchase if service is not promptly available – typically, within 20 to 40 seconds. In response, Coles has introduced Computer Vision technology to monitor deli counters. This technology is programmed to alert team members when a customer approaches, ensuring that assistance appears just when they need it.
By harnessing these AI solutions, Coles is not only enhancing operational efficiency but elevating the shopping experience, affirming its commitment to customer service excellence through innovation whilst acting on its commitment to responsible use ensuring these solutions cannot identify people or collect personal information.
A digital backbone of data and AI
Coles developed an edge computing platform it calls the ‘Intelligent Edge Backbone’ (IEB), using Microsoft Azure Stack HCI, Azure AI and ML. This solution is a global-first in retail to be deployed this way and at this scale.
Retail environments present distinct challenges for edge computing due to the vast and dispersed nature of operations. Coles operates over 1,800 sites, each representing a node at the edge of a complex network. It needed a platform that could support the significant AI workloads of large-format supermarkets. Prior solutions focused on isolated task-specific or site-specific use cases, but Coles’ vision required a unified platform that could centralise data and provide clarity and consistency across its entire operation.
The IEB serves as this central plane, enabling the collection and management of data from AI models across Coles’ network. This allows real-time operations such as queue monitoring, assisting customers through fresh produce recognition at scales, and enhancing service at point-of-sale systems. By integrating event detection with responsive notifications to team member devices, checkout systems, or even in-store speaker announcements, the IEB provides a seamless flow of information that translates into immediate action where necessary.
“The IEB is effectively our central nervous system, connecting all of the technological infrastructure we have in stores to a central plane that manages events, triggers alerts and recommends actions. It also collects a wealth of data from which we can extrapolate insights that each store can action in customised ways that work best for them,” said Roslyn Mackay, Head of Technology Innovation at Coles Group.
The IEB’s value extends beyond the immediacy of customer-facing services. It also enables insights and metadata that inform Coles’ strategic, longer-term operational decisions. Additionally, it offers valuable intelligence for planning and rostering. That’s helping Coles to align team member schedules more efficiently with anticipated peak times, to take just one example.
Turning NPS scores into valuable insights
The Finance and Operations team has been on a digital insights journey for the past two years, looking at how to build a best-in-class commercial governance and support network for 850 stores.
“The whole intention behind the Operations team’s digital insights journey is to be able to give store teams the right information at the right time to make their jobs easier. We also want to understand the impact of and relationship between our operations and the customer experience,” Rich Walker, GM of Finance, Operations & Sustainability at Coles Group said.
The team receives thousands of customer surveys per week, where each customer gives their shopping experience a score. The company also collects thousands of pieces of verbatim feedback every week. While the scores were easy to digest and analyse, the team found that there was often a disparity between the scores and the actual customer sentiment.
“We’d get a 9 out of 10 score, but then have the customer saying things like, ‘I couldn’t find a spot in the car park’ or ‘my favourite brand of cereal wasn’t stocked’. We realised that we were missing out on the richness of the insights in that verbatim feedback, simply because it’s not humanly possible to analyse the vast number of pieces of written feedback every week,” Rich Walker said.
Coles turned to generative AI to supercharge the semantic analysis of this qualitative data. The team built a model using Azure OpenAI’s LLM capabilities, Microsoft Fabric and PowerBI to ingest, analyse and generate insights from the data. It reads the comments and can effectively summarise the top three things each store needs to work on, as well as categorise the comments into 30-40 semantic themes, including product quality, availability, wait times, prices, team friendliness and more.
The Operations teams can then overlay contextual information around time of day, store location and even rostering and compare this against the customer sentiment analysis to get a comprehensive understanding of how operational decisions and factors affect the customer experience.
Looking ahead
Coles is hugely optimistic about the potential AI has to revolutionise retail.
“AI has given us the extraordinary ability to derive insight at a deeper level than ever before. And the excitement around the technology is electrifying. We are AI-ready and looking forward to what the technology can do for our people, but we are also moving with caution and a great sense of responsibility in how we use data and artificial intelligence,” said Silvio Giorgio.
Coles is also keen to explore how AI can continue to help improve its operations to provide safer and more inclusive shopping experiences that reflect the diverse needs of its customers.
