The Star spoke to Saqib Shaikh about his Artificial Intelligence application, the Seeing AI that enables visually impaired people to see the world. Read the print version here
THE surest thing about Saqib Shaikh is his clarity of vision – to enable not just the visually impaired to experience what most people take for granted, but ultimately others as well.
To those yet unfamiliar, through his Seeing-AI app, the visual world has become an audible experience to blind people.
Point your phone’s camera at (or hover it) over a spot and you’ll hear a description of exactly what’s out there. The app recognises friends, describes the people around you including their emotions, and scans and reads texts out loud as they come into view (including pages off a book, letters, envelopes and signboards).
The app also guides users. When paying with cash, the app identifies currency bills. When looking for something in the pantry or at the store, using the barcode scanner with audio cues will help you find what you want, and with some products you will also get additional details. You can even hear descriptions and images in other apps like Twitter by importing them into Seeing-AI.
It is perhaps obligatory to reveal that the 35-year-old London-based Microsoft software developer is himself visually impaired, having lost his vision at seven. He was in Kuala Lumpur for the very first time as a panelist at the Future of Work, Workplace, Workforce Conference 2017 organised by Talent Corporation Malaysia Bhd.
“The spectacles are more difficult to manufacture and are costly, whereas the app is free of charge. We wouldn’t want to make something useful that is limited to only a few people; that defeats the whole purpose,” he says.
The app captures mainly images, though a video-like feature is used for text-reading and face-recognition. “If I want to train the phone to learn about you, it just takes a few photos, so that whenever it sees you it would say your name.”
One of the unique features of the app is a computerised voice that tells you what exactly you have touched on which part of your smartphone screen.
It is only compatible with Apple iPhones for the time being and is currently available in nine countries: the United States, Canada, Australia, New Zealand, Britain, Ireland, Hong Kong, Singapore and India.
Saqib, who has been with Microsoft since 2006, says the response to his app from the visually impaired community has been “incredible”.
“I hear people say things like, ‘I never realised how much writing there was around me until I held up the app and it just read all the writing.’ People find that very empowering. Maybe it is identifying the different items in a salad bar, for example, as one user said.
“Or, as someone else said, she couldn’t do the cooking for the house and had to get help from her husband to find the boxes; however, with Seeing-AI, she could scan the product and independently move it around.
“One guy who works in a school has mounted a phone above his desk, and it will tell him when students are walking through the door and who is coming. Again, I say it is incredible that everyone in their own lives has their own little story,” he says.
The humble and affable software developer is quick to credit others, insisting he is building on the work of many scientists who had worked on this for years, and that this app was the product of his team of four.
Being visually impaired, Saqib is particularly passionate about accessibility and universal design, and serves as an internal consultant for teams including Windows, Office and Skype.
In a reflective moment, Saqib says he had kept his disability a side thing for a long time. “But I thought to myself, ‘What if I bring this to the fore and start giving back to the community?’
“We’ve released the first thing. But this is a very long journey in that the technology keeps getting better every year; it’s much better than the year before.
“It’s very possible that I might soon focus my attention on people of other disabilities. Across the company, there are many initiatives centred on different disabilities.
“One of the things Microsoft has been keen on is creating an inclusive workforce and practising inclusive hiring – that means that we’ve got people with all sorts of abilities within the company. They have their own passions and expertise. Seeing-AI has been my passion and I was able to bring it to market through a team of great engineers and scientists.”
On the current state of inclusiveness and accessibility of the disabled in the workforce, Saqib says, “I can’t speak for every country, but based on what I know, the first thing is we need to get the word out, because people don’t realise that so much is possible. The World Health Organisation (WHO), for instance, did this nice thing where it changed its definition of disability from being about a person to being about both a person and their environment.
“So it is not about someone saying ‘I’m disabled’, it is that in this environment I’m disabled, and if you improve my environment then the boundaries can be gone. When you have the right environment, there’s nothing to stop a disabled person, which is the message to get out there.”
As to the possible steps to improvement, Saqib says it is going to be different for everyone.
“So give it a go and, who knows, you may be surprised because if you have that diversity of people with different ways of solving problems, they could be a really valuable asset to your company. Don’t think about whether this might work or not; just give it a go,” he says.