How one Microsoft mom inspired health care companies to embrace the life-saving potential of AI

Melissa Mulholland had no idea the baby she was carrying might not survive. If she hadn’t gone to a high-risk clinic for an ultrasound, and if that ultrasound hadn’t been reviewed by a doctor with expertise in a certain area of fetal care, the outcome could have been much different.

Her ultrasound showed a fetal abnormality called posterior urethral valves (PUV), a congenital problem that is sometimes missed in reviews of ultrasounds unless doctors are trained to look for it, which they generally are not. PUV, which affects 1 in 8,000 males, means that extra tissue obstructs the baby’s bladder, causing a reverse flow of urine that can damage other organs, and can be fatal.

At 16 weeks, Mulholland, director of business strategy for One Commercial Partner (OCP), which helps enable Microsoft partners’ success around the world, underwent a risky procedure. Doctors inserted a catheter in utero to divert the baby’s urine, and chances of success were not guaranteed.

“When I was given this horrible news that my son would not likely make it, and that we would have to terminate early – you can imagine how devastated my husband and I were,” she says. “And I consciously made the decision that, look, psychologically I have to do whatever I can do to know that I’ve tried everything to save this child’s life. And we were fortunate that it worked.”

Through working with Microsoft partners to help create their cloud experiences, she thought with advances coming quickly in artificial intelligence (AI), could some of the companies she works with use the cloud and AI to help solve some of the many health care problems out there?

She made a vow to her infant son, Conor – and to herself – to seek that kind of change by sharing her story with companies that were interested in hearing it.

I have to do whatever I can do to know that I’ve tried everything to save this child’s life. And we were fortunate that it worked.

Crayon, a Norwegian company, was one of them. The company started out as a service-oriented software business, grew by building service offerings designed to guide clients end-to-end in making use of the cloud, and five years ago, embarked on its own AI journey.

“Melissa posed the question to us about what are some of the other ways that AI can bring forth the next frontier when it comes to health care? And it got us thinking more about health care issues or cases that we could get involved in.”

“Melissa’s personal experience got us looking at all aspects of the AI health care field. She was full of excitement and enthusiasm and good ideas,” says Crayon co-founder Rune Syversen.

Three people on Inspire stage

Tim Huckaby, left, of InterKnowlogy, and Rune Syversen of Crayon, with Melissa Mulholland on the stage at Microsoft Inspire 2019.

Since then, Crayon began using AI and machine learning in various health efforts. Among them: Working with Norway’s national hospital, Oslo University Hospital, to help augment doctors’ expertise in colon cancer screening by using advanced imaging to detect asymptomatic polyps with possible invasive cancer to make the treatment process faster; based on Azure Kinect, developing a way for nurses to monitor a hospital patient’s movements in bed so if a patient is about to fall, nurses can get to that patient before it happens; and using AI to learn more about cardiac arrest events caused by a symptom known as hypertrophic cardiomyopathy, or a thickened heart wall.

“I’m very proud of what our team has been able to accomplish in such a short time, and how we’ve been able to help doctors and nurses potentially save lives,” Syversen says.

For its work, last year Crayon was named Microsoft global AI and Machine Learning Partner of the Year.

I’m very proud of what our team has been able to accomplish in such a short time, and how we’ve been able to help doctors and nurses potentially save lives.

Another Microsoft partner, InterKnowlogy, was also touched and affected by Mulholland’s story.

Tim Huckaby, founder and chairman of the Carlsbad, California company, first met Mulholland through email when the two were working on a project together for one of Mulholland’s customers. InterKnowlogy typically builds enterprise software for other companies, including many that use AI and computer vision.

When Mulholland learned that AI played a role in the company’s work, she told Huckaby about Conor and her frustrations about there not being an easier way to diagnose PUV.

“We got to talking on a personal level. We immediately connected because I have a severely disabled brother, and we became friends,” Huckaby says.

He decided to see if he could “whip up a quick prototype” to recognize PUV in ultrasound images by using Azure Cognitive Services Custom Vision, which is an AI application programming interface (API) for computer vision to detect and recognize objects. Huckaby asked Mulholland to send him Conor’s ultrasound. Then he searched the internet for publicly available photos of normal ultrasounds, as well as ultrasounds that show PUV.

Man looks at computer

Tim Huckaby, founder and chairman of InterKnowlogy, created a prototype” to recognize PUV in ultrasound images.

I e-mailed Melissa at 3 in the morning, and said, ‘I’ve figured it out, it’s awesome, you gotta see this!

“The issue was that if a physician was trained to see PUV they caught it every time.  But if not trained, it could be missed.  My hypothesis was that if I trained the machine learning model correctly, then computers are going to just nail it 100 percent of the time.”

He started working on the prototype at 3 in the afternoon. “Look, I’m beyond my production programming years; I’m overhead now,” he jokes. “But it was one of those rare moments where I dug in on some technology and wouldn’t give up until I figured it out.”

And he did 12 hours later. “So I e-mailed Melissa at 3 in the morning, and said, ‘I’ve figured it out, it’s awesome, you gotta see this!’”

She did, the very next day via computer presentation and was thrilled. The two shared their experience in 2017 at Microsoft Inspire, the annual conference for Microsoft partners. And Huckaby has since added AI and health care to his company’s app repertoire.

His fervent hope, as is Mullholland’s, is that a health care company will want to fully build out a PUV detection line of business. “InterKnowlogy builds custom software for companies; we don’t typically retain the intellectual property (IP) of what we build,” he says. “I would love to give that IP to a company to run with it.”

Since then InterKnowlogy has been working with a large health care organization on computer vision projects to detect cancer.Family of four

The Mulholland family at home: Kyle, Conor, Melissa and Emma. (Photo by Scott Eklund/Red Box Pictures)Conor, now 5, is already benefitting from AI in his life. After undergoing 12 surgeries by the age of 2 to deal with the effects caused by PUV and related issues, he was diagnosed with autism at age 3, and has had difficulty speaking. He is working with speech therapists, and is also benefitting from an app called Helpicto that uses AI to convert spoken words into a series of images.

Created by French company Equadex, the app, too, came from the heart – several of the company’s employees have personal experiences with autism. Equadex created Helpicto using Azure Cognitive Services and the Microsoft Azure cloud platform.

“It’s very hard to have a child not be able to speak or communicate with you,” Mulholland says. “He was 4 when he first said, ‘Mama.’ When he did, it was amazing to hear it.”

Conor also learned how to say “Dada” and “Amma,” or Emma, for his 7-year-old sister, who dotes on him.

“It’s really sweet to see how caring she is and how much she wants him to be successful,” Mulholland says. “She helps him practice on words. I can see her being in a field of study someday that’s very focused on helping others, whether it’s as a teacher or doctor, or something along those lines. Her life will be forever changed because of having a brother like him.”

Mulholland says she is “humbled” by all the support she has had – from her husband, Kyle, an accountant who stays at home with Conor, to Microsoft for giving her a platform to tell her story, to the companies that want to hear and embrace it.

“I always encourage people, ‘Don’t pigeonhole yourself, think of ways that you can really harness technology to drive greater good, because sometimes those solutions are right in front of you,’” she says. “And imagine how great of a world we could live in if we had more stories like this.”

Top photo: Melissa Mulholland with son Conor. (Photo by Scott Eklund/Red Box Pictures)