AI for Accessibility
How AI is making the digital world a more accessible place
There are more than a billion people with disabilities in the world, that is one in seven individuals. An estimated 70 percent of these people have non-apparent disabilities, which means they go largely unnoticed and hence are not often addressed. Every one of us will face some type of disability at some point in our lives, be it temporary, situational, or permanent.
“Technology can empower people to achieve more, help strengthen education opportunities, and make the workplace more inviting and inclusive for people with disabilities. And with more than one billion people with disabilities in the world, Microsoft believes accessibility and inclusion are essential to delivering on our mission to empower every person and every organization on the planet to achieve more,” says Jenny Lay-Flurrie, Chief Accessibility Officer at Microsoft.
Take for instance, Windows 10, which offers several built-in capabilities that make it easier for people with disabilities to access content. For example, the Ease of Access settings in Windows 10 offers features like Narrator that reads aloud text, Magnifier that enhances the size of text and images, and a high-contrast mode to assist users with vision impairments.
For Nidhi Jain, Program Manager – AI Products Cortana at Microsoft India, User Interface (UI) designs are easier to see with more distinct colors in the high-contrast mode. Jain was diagnosed with retinitis pigmentosa at the age of 17, a degenerative, incurable genetic disorder that causes varied degrees of loss of peripheral vision and night blindness.
“Product designs have to be pixel-perfect. Comprehending images and designs are a challenge for me since I have a partial visual impairment. I work around it by using a high-contrast black mode which helps the text and images stand out,” she explains.
For people who are deaf or hard of hearing, the closed captions feature in the Ease of Access settings makes the device easier to use without sound by displaying audio as text. Users can also choose to show audio alerts or notifications visually.
Applying AI to improve accessibility
We believe that AI can be a force multiplier in unlocking solutions to some of the biggest challenges facing people with disabilities. We’re applying AI capabilities to assist and empower people with disabilities, helping make their workplace and everyday life more inclusive.
Here’s a look at some of our AI-enabled services, apps, and features, that are helping people with disabilities unlock their potential.
Intelligent Alternative Text in Word and PowerPoint: While alternative text started primarily as an attribute in HTML code to provide image context to search engine crawlers helping them index an image properly, we are leveraging AI to make visual content more accessible for people with visual impairments. Using the same technology that powers PowerPoint’s Designer feature, Microsoft Computer Vision Cognitive Service, users can now get automatic alternative text for images in both PowerPoint and Word.
For Shriram Parthasarathy, Social Media Lead at Microsoft India, intelligent alternative text is an invaluable feature. Diagnosed with macular degeneration during early childhood, he uses intelligent alternate text while running Microsoft’s social media and digital networks.
“Being a marketer, one thing you cannot live without is the presence of visual content, both in terms of static images and motion graphics. In my daily routine, one of the biggest challenges is interpreting images and graphics for what they are and how they will end up. Intelligent image descriptions help me interpret every image,” he reveals.
Seeing AI: Apps like Seeing AI have unlocked new capabilities for people with low-vision or blindness helping them navigate their surroundings like never before. Powered by Microsoft’s Cognitive Services APIs, Seeing AI essentially leverages machine vision and text-to-speech systems to recognize and understand the facial expressions of friends and family, read out sentences, identify colors, currencies and more.
Apart from the obvious use cases, Shriram also relies on Seeing AI to access images on Twitter. “I use Seeing AI to interpret images on tweets as they happen without any dependency. Even when I’m on the move I’m able to pick up an image and use Seeing AI to describe it for me,” he reveals.
Dictate: A helpful feature in Windows 10, Dictate enables hands-free typing. Powered by Microsoft’s speech-to-text technology, Dictate has been making it easier for people with low or no vision to type their emails, write blogs and more. The feature works in any text field either in Windows 10 or in an app, which users can invoke by pressing Windows key and H simultaneously.
Jain often uses it to dictate her responses to emails, instead of using a keyboard to type in every letter. “I think Dictate is a powerful tool, though it was not developed specifically for someone with a visual impairment. I use it because I can just speak to my computer and it types everything on the screen. I respond to a lot of emails using Dictate, especially when I’m having a tiring day,” she adds.
RELATED: Design for disability, empower everyone
Live captions and subtitles: We realize the growing need for captions. Not only do they empower those of us who are deaf or hard-of-hearing, but everyone benefits when captions are included—from diverse groups and organizations where multiple languages are spoken to helping people in a loud room from being left out, the benefits are endless.
Live captions and subtitles are now available in Skype and PowerPoint. Skype users can turn these on for a single call or choose to see them on every call they are part of, whether it’s a one-on-one or group chat.
PowerPoint users will also be able to display captions and subtitles in the same language or in a different one with Microsoft Translator.
In addition, we also announced that live captions and subtitles are coming to Teams, helping to make your meetings more inclusive for all attendees.
Microsoft Stream also auto transcribes the audio using our speech to text capabilities, which makes it easier for people who are deaf or hard of hearing to access video content.
Background blur in Teams and Skype: What is now considered to be an important feature for people who work remotely, background blur in Teams and Skype actually originated from a Microsoft engineer’s need to be connected to her parents in India.
Swetha Machanavajhala, deaf since birth, had just moved to the U.S. and was trying her best to stay in close touch with her parents back home, calling them on Skype every week. But their internet connection in India was poor, and she struggled to read their lips over the glitchy video. She always had to ask her parents to turn off the lights in the background to help her focus better on their faces.
“I kept thinking, ‘Why can’t we build technology that can do this for us instead?’” Machanavajhala recalls. “So I did.”
It turned out her background-blurring feature was useful for privacy reasons as well, helping to hide messy offices during video conference calls or curious café customers during job interviews. So Machanavajhala’s innovation was integrated into Microsoft Teams and Skype.
Tell Me: One of the challenges that people with low vision face is to reach the command they need, thus slowing them down. Tell Me lets you quickly access commands in several Office 365 applications without navigating the command ribbon. You can use Tell Me to assist with formatting, discover the difficult-to-find capabilities and even get scoped help in Office 365 using everyday language. Jain vouches for this feature in her daily work life.
“As a program manager, I’m expected to create long product spec documents, which must be formatted properly for designers, coders and everyone else in the team. With a simple ‘Alt + Q’ on the keyboard, the Tell Me tool lets me add bullet points, change the font and do everything else I need to make a presentable document using Microsoft Office,” she reveals.
These are just a few examples of how AI is revolutionizing accessibility. Given the right tools, anyone – be it a person with a visual, hearing, speech, mental, cognitive or mobility-related disability – can achieve more. You can find out more about our accessibility tools for people with other disabilities here.
Contributing to a more accessible world
At Microsoft, our commitment to accessibility is not limited to our products. We are supporting developers, NGOs, academics, researchers and inventors to bring their accessibility ideas to life. The AI for Accessibility program is an endeavor to provide access to cutting-edge Microsoft Azure cloud computing resources and make it easier to bring accessible products and services to the market.
You too can build on the latest advancements in AI and Machine Learning to develop accessible and intelligent solutions that see, hear, speak, understand and better interpret people’s needs.
Join us in our mission to empower people through inclusive technology. Apply for an AI for Accessibility grant here.