China has the world’s largest smartphone market. India has one of the world’s fastest growing mobile markets. Asia has more than half of the world’s population.
The continent is a critical market for technological innovation, a driving factor for a new initiative called Hack4Asia at the 2015 Hackathon, a company-sponsored, three-day event organized by The Garage. The initiative encourages all employees to develop products that empower Asian customers with new technologies.
“The Hack4Asia initiative is an invitation to everyone in the company to think about Asia. Hack4Asia seeks to foster ideation on how Microsoft can delight Asian customers and drive innovation for Asian markets,” says Elaine Chang, chairperson of the Microsoft Asian Employee Resource Group, which is driving Hack4Asia in partnership with Microsoft China, India and Israel.
This year, more than a thousand teams from mainland China, India, Israel, Hong Kong and Taiwan are part of Hackathon, highlighting Microsoft’s innovative strength in Asia. For many teams, insight into local cultural values and consumers shapes their hacks, but a desire to help all people lead productive, fulfilling lives is what drives them.
Here’s a look at a few of their projects.
Sponsored by Harry Shum, executive vice president of Microsoft’s Technology and Research Group, TalkEasy transcribes speech into real-time text to help people with hearing loss communicate. Based in Redmond and Microsoft Research Asia in China, the TalkEasy Hackathon team is using Project Oxford’s speech APIs, which provide state-of-the-art algorithms to process spoken language.
TalkEasy grew out of Shum’s inspiration to help his father, who is hard of hearing and doesn’t like to wear a hearing aid.
“I thought it will be very helpful to build a simple tool that he can use with his mobile device to see nearly simultaneous speech transcripts as he talks to others,” Shum says.
The easy-to-use application will have enlargeable text and adjustable scroll speed for older users. It will transcribe conversations with multiple speakers and in various languages, including English, Chinese, Japanese, Spanish, French, Italian and German.
Like other Hack4Asia projects, TalkEasy was inspired by the values of its team, which include a respect for elders. But it encompasses a global goal of helping people connect with loved ones across generations.
“Not feeling like you can connect with people and really communicate is difficult, so empowering people to do that is very important,” says Allison Light, a program manager in Microsoft’s Technology and Research Group and a member of the TalkEasy team. Other members of the team include Arthur Jiang, Jiaping Wang and Yu-Ting Kuo.
Cortana vision for Visually Impaired
This project, being developed in India and Redmond, integrates Cortana’s intelligence and voicing abilities with street navigation, emergency support and real-time bus data to help people with visual impairment get around. The mobile app will work in English, Hindi and a local Indian language that hasn’t yet been determined. It will also connect to a walking stick equipped with a sensor.
A group of software engineers in India plotted out the idea to build an app with Cortana for people with visual impairment, says Ravishankar Sanjeevaiah, a development and operations engineer in the Marketing and Partners group in India.
Shree Lakshmi Rao, a Bing Studio designer, also had a similar idea and pitched the team to take the project to the next level. Rao had studied how technology can help transit users with visual impairment while at Carnegie Mellon University’s Robotics Institute.
“We’re specifically focusing on a user group that isn’t always the target of a lot of technology,” says Rao, who works in Redmond. “We’re trying to support and empower them to do more.”
The seven-person team is also exploring crowdsourced information to help people navigate in developing areas that may lack sidewalks or digital maps. They’re also studying how the app can help build a similar system for feature phones, because many people with visual impairment in India don’t have a smartphone.
“The idea is, the more input and usage Cortana gets, the smarter she gets,” says Rao. “Is there a way we can harness that?”
The Band-It team in Israel puts an innovative twist on the Microsoft Band by taking the health and wellness device into a productivity setting. With the device’s built-in accelerometer, it detects hand gestures to control Office applications in different scenarios.
“The most obvious one would be while doing a presentation – using ‘band gesture’ to replace that annoying clicker in your sweaty hands,” says Keren Master, a senior program manager at Microsoft Technology and Research’s Advanced Technology Labs in Israel.
Another scenario: Using Band-It to navigate your inbox without touching your mouse or keyboard, and while leaning back in your chair as you make your way through a mountain of email.
Master specializes in new natural user interface (NUI) experiences, and her eight-person Hackathon team spans different groups, disciplines and skills. It includes women, men, engineers, researchers and program managers, from Office, Bing and Technology & Research teams.
“We are taking an existing product, the Microsoft Band, and exploring how we can expand its usage to new domains to empower its users to be able to do more,” says Master. The team and Microsoft Israel are partnering with Hack4Asia to increase their impact.
WeChat Office hacks in China and Redmond
Microsoft China’s Office Innovations Team has several hacks that give consumers powerful, integrated tools for productivity and socializing. The popular Chinese messaging app, WeChat, has almost 600 million global monthly active users who not only message with the app; they also play games, pay for dinner, shop online and book flights with WeChat.
“WeChat has been dominating the IM market in China. Almost every smartphone gets WeChat installed,” says Bo Gao, a senior program manager with Microsoft Office in China. “All the mobile apps in China do not want to miss this channel to promote their usage. Neither do we.”
Last year, the Chinese team announced an education project that uses WeChat as a socket to bring Office 365 into common school scenarios, including parent messaging and photo sharing.
This year, Gao is leading a Hackathon team to build a common layer between Office 365 and WeChat to help developers create customized solutions. Another hack, led by software engineer Raymond Xue, will add cool new features to a WeChat2OneNote tool. A third project will improve the experience of buying Office 365 from the WeChat platform, says software engineer Ginger Jiang.
Chang, chair of the Asian Employee Resource Group and a senior program manager in Redmond, is also leading a hack team on WeChat2Sway, to make collaborative storytelling easy and visually compelling for WeChat users.
Kisan 4 You
From childhood, many Microsoft employees in India have seen their farming parents face day-to-day problems due to a lack of advice, access to direct buyers, knowledge of commodity prices and information on government programs.
To help bridge that gap in communications and knowledge, a group of employees is working on the Kisan 4 You Hackathon project in India. Its aim is to take technology to the nation’s farmers, 87 percent of whom are expected to use smartphones in the near future.
Network connectivity is an issue, so the team is building its app to work in disconnected mode. It’s all part of a goal to help India’s farmers tap into the information age and give the country a tool to further economic stability.
Skype for Business MoM Generator
Anyone who sits in long meetings will appreciate this Skype for Business add-on that generates minutes of a meeting (MoM). It comes from a team of IT software engineers in India, who have sat in their share of lengthy Skype meetings with collaborators in Redmond.
“A consistent pain point is that one person needs to talk and contribute to the meeting, as well as keep track of who is saying what, what are the main points and what are the final features we are working on. This puts a strain on that person,” says Ujjwal Sen, an engineer with the Enterprise Services Business Intelligence Team in India.
The application uses machine learning and speech recognition technology from Microsoft Audio Video Indexing Service (MAVIS), developed by Microsoft researchers. Available on the cloud as the Azure Media Services Indexer, the tool extracts key words from a recording to generate a transcript of main points. Among Sen’s challenges: Accommodating multiple speakers and accents.
“We want to help people be more productive in meetings,” says Sen. “Plus we are hoping we can showcase Microsoft’s eminence and technology power in the machine learning space – specifically in the speech recognition and speech intent analysis domain.”
Lead photo: Pictured clockwise from the top left are 2015 Hackathon teams Cortana vision for Visually Impaired, Band-It, Skype for Business MoM Generator, Easy buy Office 365 at WeChat, and WeChat2OneNote.