DrumBeat.ai project taps AI and cloud to tackle Indigenous hearing challenge

Aboriginal and Torres Strait Island children are three times more likely to suffer from otitis media (middle ear infections) and twice as likely to have long-term hearing problems than non-Indigenous children in Australia.¹ If left unchecked, ear disease can have devastating impacts throughout a child’s life.⁴

But limited access to Ear, Nose and Throat (E.N.T.) specialists is a real challenge for communities – especially for Aboriginal and Torres Strait Islander children in rural and remote areas. The prevalence of chronic ear disease in such children is nearly ten times greater than the level which the World Health Organisation (WHO) considers a massive public health problem, requiring urgent attention.⁵

When it comes to chronic ear conditions the disparity is even starker for Aboriginal and Torres Strait Islander children in rural and remote areas. The prevalence of chronic ear disease in such children is nearly ten times greater than the level which the World Health Organisation (WHO) considers a massive public health problem, requiring urgent attention.⁵

Being unable to hear properly can drastically affect a child’s ability to develop speech and language skills, learn and interact with family and friends. It can contribute to behavioural problems and limit future employment opportunities, leading to lifelong challenges.

That’s why checking rural and remote Aboriginal and Torres Strait Islander children’s ear health and hearing is so important. During routine health check-ups a health worker, nurse or doctor will examine a child’s ears using an otoscope (ear camera) and if there appears to be an issue will capture an image that can be further analysed.

But Australia faces a shortage of Ear, Nose and Throat specialists which can lead to delays in diagnosis and treatment, particularly if the child lives outside a metropolitan area.

Now a landmark project, called DrumBeat.ai (www.drumbeat.ai), is exploring how artificial intelligence can be used to interpret and triage the images – allowing at risk children to be quickly identified regardless of where they live.

Dr Al-Rahim Habib is an Ear, Nose and Throat Surgery Unaccredited Registrar and a PhD candidate at the University of Sydney, leading the DrumBeat.ai project, supervised by Associate Professor Narinder Singh, Head of ENT at Westmead Hospital and a clinical academic at the University of Sydney. The aim of the DrumBeat.ai project is to empower front line health workers who might not have the expertise or experience at recognising ear disease or determining when urgent referral needs to be expedited – but can now get support via the digital platform.

To address the issue, the DrumBeat.ai team has been working with the Azure FastTrack team to develop a Microsoft Azure based solution that uses Azure Custom Vision and a machine learning model, trained using 20,000 existing otoscopic images, which can then analyse new images on the fly.

Analysing otoscopic images is notoriously difficult; early results from the project are highly encouraging and show that the DrumBeat.ai algorithm can differentiate normal from abnormal eardrums with an accuracy comparable to ear disease experts.

Associate Professor Narinder Singh, chief investigator on Drumbeat.ai says; “Having grown up on my family’s farm in Griffith, country NSW, I have first-hand experience of the difficulty in accessing timely healthcare in the bush. Any program that accelerates the speed at which at-risk children can be treated can have a huge impact for patients, care providers and the community.

Because the DrumBeat.ai analysis can be done in the cloud, there may be no delays waiting for a specialist to become available in a particular location, meaning children most at risk can be triaged to receive priority care.

This is particularly important for Aboriginal and Torres Strait Islander children living in rural and remote parts of Australia where access to healthcare specialists can be especially challenging – never more so than during a global pandemic when border closures and snap lockdowns can restrict access to health services.

Even before the pandemic, taking a child with a hearing problem to see a specialist who may be located hundreds of kilometres away was a major challenge for remote communities. DrumBeat.ai brings the expert to the patient.

While developed in Australia, Dr Habib says similar problems exist overseas especially in regions such as India, sub-Saharan Africa and among First Nations people in the US and Canada.

DrumBeat.ai has benefitted from the world class collection of images that has been made available to train and finesse the algorithm. According to Dr Hemi Patel, a Northern Territory Ear Nose and Throat Surgeon who has been instrumental in establishing the image database; “Data is the key to any diagnosis. DrumBeat.ai has used an existing and world-class collection of analysed images to refine and train its model to be able to identify abnormal eardrum images with a high degree of accuracy, and to then triage patients for treatment. As this work proceeds our knowledge and data collection will expand and that, in turn, will allow further refinement of the model, leading to even better outcomes over time.”

Dr Nic Woods, health industry executive at Microsoft Australia and New Zealand, says DrumBeat.ai is an example of how AI infused digital platforms and telemedicine can accelerate diagnosis and opportunities for treatment.

“Microsoft is delighted to support Dr Habib and the DrumBeat.ai team in this important work with access to our Azure Fast Track engineering experts and an AI for Good grant.

“The pandemic has accelerated digital health initiatives across the world, and demonstrated the power of telemedicine. DrumBeat.ai takes this to the next level, infusing AI to interpret images and ensure that those in need receive the medical attention they need to ensure long and healthy lives.” Because the DrumBeat.ai analysis can be done in the cloud, there may be no delays waiting for a specialist to become available in a particular location, meaning children most at risk can be triaged to receive priority care.

Microsoft caught up with Dr Habib to learn more.

Microsoft (MS): Can you explain in a nutshell what this system is designed to do?

Dr Al-Rahim Habib (ARH): Aboriginal and Torres Strait Islander children have the highest rate of ear disease in the world. Childhood ear disease can cause hearing loss, speech problems, poor academic performance, and unemployment. Ear, Nose and Throat specialists (or “ENTs”), are difficult to access in rural and remote areas. Most ear exams are performed by local community health care workers who can look at the eardrum using a specialised camera or otoscope. However, ears are complex and often it takes many years to appreciate the differences between normal and abnormal eardrums and recognise urgent conditions that require a specialist referral. We’re creating an AI tool that can autonomously interpret otoscopic eardrum images to assist health workers recognise whether a child’s ear is normal or abnormal, predict the likelihood of developing hearing loss, and how urgently they need to be referred to an ENT specialist.

MS: How can digital platforms help?

ARH: Digital platforms have the potential to revolutionise the delivery of health care services. For example, telemedicine is an important component of providing ENT services to people who live in rural and remote areas. Tele-otology or telemedicine related to ear health, allows ENT specialists to review digital eardrum images and hearing tests from a distance to establish management plans. This is valuable because people who live in rural and remote areas often face challenges to access ENT specialists by travelling long distances, experiencing long wait times, may be unfamiliar with the health care system or engaging with services they may not find culturally appropriate.

Digital platforms can help us overcome some of these challenges by enhancing the type of services local health care workers, nurses and doctors can provide to rural and remote communities. We can develop AI tools using data previously reviewed by experts to provide a resource for frontline health care workers to inform clinical judgement and enhance decision-making for the children they are reviewing. Digital platforms can harness existing telemedicine infrastructure and enhance service capabilities using established community partnerships.

MS: What’s involved in the ear tests?

ARH: An ear test is a comprehensive assessment that includes inspection of the ear canal and eardrum using an otoscope or ear camera. We assess the eardrum for redness, swelling, infection, holes, discharge, abnormal growths, inserted objects, wax, or other signs of disease. Interpreting otoscopic eardrum images can be challenging. It takes many years of practice to identify abnormal eardrums, including “glue ear” (otitis media with effusion), infections (acute otitis media) or chronic otitis media and cholesteatoma. It is important to accurately interpret eardrum images so you can treat ear disease early, initiate urgent referrals if required and minimise the risk of long-term hearing loss. Early recognition of ear diseases like cholesteatoma is critical to protecting a child’s hearing and preventing major complications like brain and bone infections.

MS: So how are you tackling this with DrumBeat.ai?

ARH: To improve recognition of ear disease, we are creating a supervised machine learning algorithm in Azure to interpret otoscopic eardrum images with an accuracy comparable to ENT specialists. We are using a database of images collected from the past ten years in the Northern Territory with ground-truth labels established by board-certified ENT specialists.

We envision this tool being used in rural and remote settings where local health care workers with limited experience can look inside a child’s ear to instantly get an idea of the most likely diagnosis, risk of hearing loss, and how urgently a referral is required. Integrating this technology into existing health care services has the potential to revolutionise how ear health is being managed.

MS: Can you describe the underlying technology?

ARH: Early versions of Drumbeat.ai use the Microsoft Azure Custom Vision service to establish a supervised machine learning image classification algorithm developed with deep learning and convolutional neural networks. We use transfer learning to harness knowledge gained from pre-trained models and tailor them to address our clinical needs. In this case, we used Custom Vision to train a classifier using otoscopic eardrum images collected over the past decade by the ENT Department at the Royal Darwin Hospital in the Northern Territory. We have a lot of work ahead of us but in the future, we aim to deliver this technology to the frontline, accessible with edge- and cloud-based processing via otoscopes, a mobile app or web applications. Depending on the location and access to the internet we envision that images can be uploaded into the algorithm for instant analysis to assist with decision-making in real time.

MS: What happens when the nurse gets the diagnosis?

ARH: We are in the early stages of integrating this technology into existing ear health programs. We are working closely with the ENT Department at the Royal Darwin Hospital in the Northern Territory and the Deadly Ears Program in Queensland. In the existing workflow, rural and remote health care workers, nurses, and doctors perform ear exams and can refer children to an ENT specialist for review. The existing system relies on the time, resources, and experience of frontline staff to recognise at-risk children. However, children may have to wait months before seeing an ENT specialist. Current telemedicine strategies have improved this gap in care but at present, frontline staff do not receive on-demand, point-of-care support. DrumBeat.ai can contribute to improving this clinical scenario. Our goal is that DrumBeat.ai will be used as an AI-assist to frontline staff by providing them the most likely diagnosis and triage category. This can prompt nurses to initiate treatment as per the Otitis Media guidelines, expedite referral, and establish a plan for follow-up.

MS: What sort of accuracy are you achieving?

ARH: Our classification algorithms are able to identify common ear disease categories – normal eardrums, acute otitis media, glue ear (otitis media with effusion), chronic otitis media and wax / foreign bodies in the ear canal. To differentiate normal from abnormal eardrums we achieve over 85% accuracy, depending on the type of ear disease. We are in the process of evaluating how these early results compare to humans with or without AI-assistance.

Recognising ear diseases like glue ear and chronic ear disease can be challenging but it is critical to maintain ear health and normal hearing. As the telemedicine programs continue to evolve, we aim to collect more images and refine our algorithms. In the future, we hope to enhance the algorithm by including more variables, such as symptoms and hearing tests, into the model to achieve higher levels of performance.

We have a lot of work ahead of us before we can confidently integrate this technology into routine clinical practice. We must ensure that algorithms such as DrumBeat.ai are accurate, valid, reliable, clinically useful, and sustainable. It’s also very important to consider the ethical implications of this technology and its role to act as an adjunct to existing services.

This is a phenomenal opportunity to apply cutting edge technology to address an urgent public health need. The goal of DrumBeat.ai is to improve the ability to recognise ear disease, triage at-risk children, and establish a pathway for them to receive care quickly to minimise hearing loss.