Tali Sarnetzky doesn’t step out of her apartment unless she’s planned her trip in meticulous detail. Blind since she was treated for a rare form of eye cancer at the age of 3, she needs to create a mental map of where she’s going before she feels confident enough to leave. Every outing that 45-year-old Sarnetzky takes requires thoughtful preparation.
“It’s like intelligence gathering,” she says. “I need to get as much information as I can about the place I’m going to.”
Even when she does have a detailed picture of the journey and the layout of her destination, Sarnetzky still finds herself relying on “the kindness of strangers” and asking for help when it isn’t clear where she is.
In recent months, however, life has changed for Sarnetzky, a part-time secretary who lives in Ra’anana, about 10 miles north of Tel Aviv in central Israel. She’s moved into her own apartment, plans to explore more of her new neighborhood and hopes to begin studies at her local college.
Her courage has been bolstered by a free app on her smartphone. RightHear is a new way-finding technology developed by a local startup that helps people with orientation challenges navigate inside buildings. Through networked beacons, app users who are blind or low vision, or with dyslexia or agoraphobia, can be fed audio pointers and details about specific locations with the environment. More than 100,000 people globally use the app in 2,000 locations, including shopping malls, municipal buildings, restaurants and even Israel’s parliament, the Knesset. It’s also being piloted at Microsoft’s campuses in Tel Aviv and Herzliya.
The app acts like an invisible guide, giving details about, and proximity to, nearby services such as elevators, bathrooms and exits, as well as stores, offices and other sites. The prompts are directional, giving guidance according to where the phone is pointed, and has an easy connection to emergency services.
“It felt wonderful,” Sarnetzky says of the first time she used the app. “What you would do with your eyes, you’re actually doing with your phone. It makes it a lot easier to get out of the house and go somewhere when you know that there are ways for you to get around on your own.”
Hear from other RightHear users about how the app has impacted their daily lives:
Lost and found
RightHear is one of a new generation of digital accessibility aids that are harnessing connected technology to improve the lives of the 1 billion people the World Bank estimates live with at least one disability.
A part of Microsoft’s AI for Good Acceleration Program, which helps startups in Israel advance their artificial intelligence (AI) solutions and helped fast-track the app’s development and commercialization, the app can be programmed for 26 languages. It has been built to integrate with other assistive technologies including Microsoft’s Seeing AI. Right Hear co-developer Idan Meir expects to add connectivity with other apps soon, including navigational systems to guide users safely in from the street.
Raz Bachar, Microsoft for Startups managing director in Israel, brought RightHear into the program which provides support, including mentoring, business and technical advice as well as resources to prepare entrants’ products for market. Bachar saw the potential of the app after it was awarded Geektime’s Bootstrap Startup of the Year Award, which he describes as the “Oscars of startups.”
“There are so many amazing startups in Israel but the one that makes you want to go above and beyond in supporting them is the one with founders who have a passion for the services they provide, the ones who are interactive with you – this is what we saw in Right Hear,” Bachar says.
“They were not only able to grab the opportunity of the AI for Good program, they were also able to leverage our AI Azure Cognitive Services,” he adds. “They have a unique solution to a great accessibility need, which aligns with Microsoft’s investment in the accessibility ecosystem.”
Companies are taking advantage of this adaptability to link into their own digital accessibility platforms. McDonald’s, for instance, has installed RightHear at its 200 restaurants in Israel. It ties into other in-store visual aids such as magnification features on order terminals that have “low-view” displays that customers in wheelchairs can access. RightHear indicates where registers, bathrooms and other services are located.
“As a company whose orders are mostly made by digital platforms, we have been impressed by the smart technology that is simple and user-friendly,” says Eyal Padan, head of digital marketing for McDonald’s in Israel.
RightHear was built after consultation with people who are blind or have low vision, but the initial spark came from a different challenge faced by Meir. He and business partner Gil Elgrably had come across the iBeacon technology that gives RightHear its way-finding capabilities seven years ago but weren’t sure how it could be beneficially used. While exploring ideas, they tested the beacons’ usefulness in a variety of public spaces. But they wasted much of their time trying to find one another first.
“I always get lost, wherever I go – that’s part of my life,” Meir explains with an embarrassed laugh. “Gil would be on one side of a mall and I on another side and it would take an hour just to meet each other.”
Meir’s ability to get lost in even his hometown got him thinking about the difficulties faced by people who live permanently with a sense of dislocation about their immediate surroundings. Meir and Elgrably, who are also based in Ra’anana, didn’t set out to create an accessibility app and initially considered incorporating the technology into a shopping coupon app, but that concept lacked something for Meir. As tech-minded entrepreneurs, they simply wanted to get a new product up and running.
The experience of developing RightHear has been a humbling one for them, illuminating issues surrounding orientation they hadn’t before considered. That was underlined when the COVID-19 pandemic made close contact with other people potentially life-threatening.
“We were looking to do something more meaningful, more purposeful, more impactful,” says Meir, a 34-year-old father of two.
The pair contacted disability groups in Ra’anana to understand how the technology could be best deployed and were connected with members of the blind and low vision community in their local area. Sarnetzky was among them.
“Gil called me and there is no other way to describe it than he interrogated me,” she jokes. “He interrogated me for an hour-and-a half. What would I prefer – distances in meters or in feet? What would be more convenient for me – if we counted the steps or we didn’t count the steps? They applied every single thing I suggested.”
From that research a prototype system was built on Microsoft Azure and piloted in a local museum. Beacons were dotted throughout the facility, networked and programmed to highlight points of interest and important services via a digital control dashboard. Test users were then asked to try it out. The feedback was not just positive, it was also imbued with emotional significance.
The artificial intelligence behind the technology worked well. When the matchbox-sized beacons were placed at strategic positions they would connect to create an augmented reality within which physical things could be located. Users could log in and receive audible orientation prompts via regular headsets.
“The first reaction from users was really, really meaningful,” Meir recounts. “It felt like I had given them a piece of the moon!”
Learn more about RightHear’s origin story from its co-founder, Idan Meir:
It’s complicated
Yardena Ifrach is the accessibility coordinator at the Open University of Israel (OU) and is proud of how inclusive the university is. Formed in 1974, the university has more than 50,000 students who are taught remotely and within 60 campuses. Among its student body are 7,000 who hail from minority Arab communities and it’s Israel’s only higher-learning institute with a female president. Ifrach says people with a disability comprise a large minority of students and staff.
“Our goal is to think about everybody, so that everyone can feel equal and have the same equal chance and equal experience,” she says.
The OU strictly follows accessibility regulations. For people who have a visual impairment, yellow, tactile floor tiles offer directional aids, and elevator controls, bathroom doors and other services are denoted with a Braille plaque. But Ifrach says their usefulness is limited.
“The tiles give some hints, but they won’t really let people know where to go – they won’t give a full orientation solution,” she says. And the Braille signage?
“It’s of limited use, not many people use it,” Meir explains. According to the data from the National Federation for the Blind, fewer than one in 10 registered blind Americans were able to use Braille in 2009. More recent data on children suggest that percentage hasn’t changed.
“Also, if you’re blind, how do you find the Braille plaques? And when room functions or services are changed, often the Braille signs aren’t.”
So, Ifrach scouted for alternative technology-based solutions. In 2016 she met Meir.
“I immediately saw the potential of his app,” she says. Soon after, the OU became RightHear’s first customer.
While most OU learning takes place remotely, it holds many in-person lectures at its campuses, especially the sprawling headquarters facility in Ra’anana, where many of the university’s 2,000 academic and administrative staff are based. It’s also Meir’s alma mater.
RightHear’s beacons are located at the OU’s six largest facilities. The beacons have proven especially useful in Ra’anana, which Ifrach describes as the OU’s “most complicated” site, with hundreds of offices, lecture halls, canteens and other services. Even sighted visitors use RightHear to find their way around the maze of corridors and hallways. Ifrach is now looking to incorporate RightHear into other digital accessibility platforms used by the university, including classroom-based technology and other navigational tools.
“We were super excited when she spoke to us,” says Meir. “Yardena and the OU are going into this new innovative approach and others are following.”
Good for business
One of Sarnetzky’s test tasks was to visit a branch of supermarket chain Shufersal that was piloting RightHear and gather a list of groceries using only the prompts from her app. She was doubtful it would work. In stores, she could usually locate product types from non-visual sensory cues, like the whir of dairy refrigerators, the smell of the bakery or the coolness of the vegetable chillers. But finding specific items was difficult.
The RightHear app changed that.
“It sent me to the right aisle, showed me how to find the right fridge and then I could find the right product,” Sarnetzky says. “I thought, ‘Holy smokes, I did it!’”
That it’s given her the confidence to go shopping more frequently highlights a key driver behind the development of accessibility technologies: they are good for users, and they are good for business. Positive customer experiences boost brand loyalty, and that applies equally to all customers.
“If you become more social and you care about people who have different disabilities it creates a good social impact, but it also provides value in the business,” says Shufersal Executive Vice President and Chief Customer Officer Zvi Baida. “You get some yield from that.”
Shufersal is the largest supermarket chain in Israel with almost 400 locations. Many serve minority and remote communities, meaning inclusion is a vital ingredient of its operating model. It tests dozens of new accessibility applications each year, but many are rejected for needing too much infrastructure or for being intrusive.
When Meir contacted the company, store managers saw the potential to make their cavernous hypermarkets less intimidating to shoppers who are blind or low vision. RightHear beacons, small and easy to install, now direct more than 6,000 users around 200 of its stores on a daily basis. Visitors are greeted with a welcome prompt that explains the stores’ product zones and offers pointers to vital services. The app guides users to product ranges and individual shelves.
In three years of use at Shufersal, Baida says the supermarket has received only positive feedback. That’s good not only for corporate perception, but also for morale within the business.
“We have a saying in Hebrew: If you save one soul, it’s like saving the whole world,” he says. “If you can help people who have less confidence, and are less able to look after themselves, then we have a really important role.”
Hear from the CEO of Shufersal about the importance of inclusive business practices:
Digital freedom
Tali Sarnetzky still draws up travel plans for even the smallest of journeys, and moving around busy streets still requires help from others. But she is thankful for the freedom the RightHear app gives her at some of her destinations.
“Independence is not doing everything on your own, it’s having the choice of when you want to do something on your own and when you wish to ask for help,” she says. “This app allows us to be really independent.”
Top photo: RightHear user