Why better AI starts with the people it often misses
Ask an AI tool for a picture of “someone at work,” and you’ll often get a person at a desk, in front of a computer, maybe holding a coffee cup. When people with disabilities appear at all, the images sometimes have been jarringly wrong: amputees with extra limbs, blindness portrayed with blindfolds, those with dwarfism sporting huge, pointy ears.
The problem isn’t intent; it’s absence. AI systems can only learn from the data they’re trained on and the criteria used to judge their outputs. For many groups — including the estimated 1.3 billion people living with disabilities — there hasn’t been enough focus on representation online to shape either.
As AI tools become part of daily life, a different approach is emerging. Communities of people connected by shared experiences are becoming more than end users; they’re bringing their collective voice and working with Microsoft researchers to help define representation, evaluate AI systems and shape development.
The well-known “nothing for us without us” principle “makes a lot of sense,” says Nicholas Kalovwe, a project officer at Kilimanjaro Blind Trust Africa in Nairobi, which is partnering with Microsoft to make sure AI-generated imagery reflects how people with vision disabilities actually live and work.
“There are insights you simply can’t gain from a boardroom or through desktop research alone,” Kalovwe says. “To truly understand people’s needs, you have to engage directly with the communities themselves.”
Who decides what gets seen
Microsoft is working with several communities to build libraries of images that depict them in a more accurate and representative way. Those groups can offer the datasets to tech companies and others for training and testing AI systems, a process that typically requires thousands of examples.
For the children and young adults Kilimanjaro Blind Trust works with, that has meant challenging assumptions about people with vision disabilities. They’re curating image datasets showing their full lives across work, school and family life in urban and rural settings, including well-dressed businesspeople and active farmers and athletes instead of beggars sitting on street corners, Kalovwe says.
Another group, global prosthetics manufacturer Ottobock, has run broad advertising campaigns in the past few years showing people with limb differences dancing, traveling and navigating daily life — part of efforts to help society see the whole person and not just the disability, says Martin Böhm, the Germany-based organization’s chief experience officer. But the group realized that as AI plays a growing role in shaping people’s perceptions, representation couldn’t stop at marketing — it had to extend to the data AI systems are trained on.
“AI can only imagine what it has already seen, and it has never seen the real life of amputees,” Böhm says.
So Ottobock enlisted Microsoft’s help to launch an initiative this month to collect images of amputees participating in common activities — not just in athletic or medical contexts but in everyday settings community members say are rarely reflected, from parenting to cooking. Ottobock formed a group of representatives with upper and lower limb differences to select and annotate images and then assess AI-generated media against those standards. Once it’s complete, the organization plans to publish the library on an open-source platform, making it available to anyone working with AI.
Importantly, the libraries are created with Microsoft’s support but are owned and managed by the communities themselves, says Cecily Morrison, a senior principal researcher at the company. It’s just one of the projects at the center of Morrison’s work with a multidisciplinary team to build AI tools that include disability communities as partners and make sure they don’t have to see themselves through someone else’s algorithm.
“The community — not researchers or brand teams or whatever — decides what counts as good representation,” Böhm says. “This editorial authority remains with them. It’s theirs.”
When images shape reality
Research in media and social science has long shown that how groups of people are portrayed shapes how society thinks about them, with direct influence on access to things like education and employment, Morrison says. AI‑generated media is quickly becoming one of the main sources of imagery online, meaning those portrayals will increasingly determine whether barriers multiply or ease, she says.
“It’s the fastest-adopted technology that we’ve had,” Morrison says. “So getting it right really matters, and what is right is not something that Microsoft should decide.”
The company can, however, provide technological scaffolding to help people get involved in improving AI systems and shaping how they’re represented by them, she says.
That work builds on steps Microsoft and other tech companies already take to prevent harm, such as training models to avoid sexual content and violence.
“It’s not enough to get rid of the really offensive things,” Morrison says. “We also have to get to good. And it’s really important that communities help define what ‘good’ is, because if we don’t know, we can’t get there.”
Evaluation as a point of influence
It’s just as important to involve communities in the process of judging how well an AI system is meeting that definition, says Sunayana Sitaram, a principal researcher working on language and global inclusion at Microsoft Research in India. Leaving the subtleties of evaluation to a narrow set of researchers and companies, who often work primarily in English, “leads to a lot of Western-centric biases,” Sitaram says.
Take health queries, for example. They’re social and cultural in nature, she says, not just medical. So benchmarks assessing an AI system’s answers need to reflect those contexts — such as a maternal-health tool knowing that while it’s popular for expectant mothers to have gender reveal parties in some countries, in others it’s illegal for medical professionals to disclose that information, so even asking about it isn’t part of the culture.
“Someone has to know that’s how things happen in different parts of the world and then go and create the system evaluation and model differently,” Sitaram says. “Given that AI is going to pretty much reach everyone on the planet, everyone should have a say in how it gets developed.”
Evaluation is one of the places where that influence can realistically happen beyond the “huge resources” and tech expertise required to actually build AI models, she says.
Sitaram’s team works closely with civil society organizations in four high-priority sectors in India — healthcare, legal services, education and finance — leaning on those groups to collaborate with the communities they serve in designing evaluation frameworks rooted in how they seek information and make decisions.
Communities aren’t all the same, and Sitaram says the point isn’t to find one voice that speaks for everyone but to build processes that reflect different experiences.
It’s a new mix of AI research and the social sciences, she says, and a necessary one.
“If we don’t even know how well our technologies are working in different contexts, how will we improve them?” she says. “If you don’t measure it for everyone, then you can’t improve it for everyone. And there are billions of people and seven thousand languages in the world, so it becomes a very big scaling challenge, but that doesn’t mean we don’t try.”
Her team is developing reusable templates that can be used to create community-centered benchmarks around the world, along with public leaderboards that other researchers and developers can use to help build more inclusive and context-aware AI models.
“We’ve shown that it is actually possible to incorporate this kind of community input into an evaluation and make it work at scale,” she says. “That’s our contribution. It’s advancing the science of evaluation itself.”
When participation affects outcomes
Participation isn’t just about how models are trained or tested. It’s also about making sure communities have real say over the information being collected from their lives and how it’s used, says Mary L. Gray, an anthropologist and senior principal researcher at Microsoft.
Equipping communities to shape the inputs as well as the outputs of AI makes it more likely they’ll benefit from the tools they help create, Gray says.
Her work examines what that looks like in practice, particularly in health and social services — environments where trust is essential and the information involved is deeply personal, from details about medical conditions to income and housing.
“Community engagement is a methodology,” Gray says, and participation is more than a box to check after technology is built.
A big part of that comes down to who controls the data and who gets to decide what happens to it. People shouldn’t only be asked before their information is used, she says; they should be able to change their minds later. That may sound obvious, but it’s uncommon — especially when data is collected by larger organizations several steps removed from the front lines of care.
Trust is built — or lost — in data
Rather than asking people to simply sign away rights to their data and have it swept into centralized systems, Gray’s team is working on approaches that help communities securely keep control of the information they collect and generate, while still using AI tools to learn from it. That matters because meaningful insights come from patterns across many patients, not from any one record in isolation, she says, and communities are often best placed to bring that information together responsibly.
Her team is working with the nonprofit flok, for example, whose app helps patients and caregivers managing certain rare metabolic disorders track the kinds of details they’re noticing in daily life — what they eat, how they sleep, how they feel from day to day. It’s information that doesn’t often show up in clinical charts but is invaluable for medical researchers trying to understand how a disease behaves.
The effort is enabling flok’s members to pool their experience so they can contribute to research on their conditions without losing control over their personal information.
The window to get it right
All of these efforts — from image libraries to community‑defined benchmarks to secure data platforms — are unfolding early in AI’s public life, Morrison says, at a moment when the technology is spreading faster than the systems meant to guide it.
Generative AI has only been in the public sphere for about three years, and so far, “people have just been trying to make it work,” she says. “Now it’s like, OK, we don’t just need to make it work; we need to make it work for everybody.”
The stakes are high, and immediate.
“We want a world we want to live in, and to do that, we have to build the right world, and we have to do it right now,” she says. “We don’t want to wait and find out in five years that we’ve got the wrong world, because backtracking won’t be possible.”
Top photo: People with limb differences are working with Ottobock and Microsoft to expand image datasets for AI training that better reflect everyday life. Photo courtesy of Ottobock. Story published on May 19, 2025.
Susanna Ray writes about AI and technology, with stories that show its real‑world impact and examine how innovation is reshaping work, business and society. She previously reported for Bloomberg News and other major international news organizations in the U.S. and abroad, covering beats ranging from politics and government to business and aviation. Follow her work on Microsoft Source.