Microsoft Hackathon 2016 winner InstaFact uses Bing knowledge graph to help people do more
Let’s say you’re writing a report on Washington state that includes historical dates and a spreadsheet of cities and population. To do that now you need to search for the info online and then copy and paste, which is tedious and slow, as you toggle between web, report and data columns.
But someday, you might be able to use InstaFact, a new solution built by a team of Microsoft employees to help writers, researchers and pretty much anyone who needs facts quickly. The Office plug-in infers the facts you want, mines a massive information repository to get them, auto-completes your facts in Word and auto-fills your data cells in Excel — all with a few simple clicks.
On Tuesday, Microsoft crowned the InstaFact team as the grand prize winner of //oneweek Hackathon 2016, the company’s annual celebration of employee innovation held last month. The Bellevue, Washington, team beat out more than 1,560 teams that competed in the event, which saw more than 3,800 projects from around the world.
InstaFact’s proof of concept integrates Microsoft’s Bing knowledge graph into Word and Excel to help people save time, work more easily and be more productive. Called Satori, the knowledge graph is an enormous repository of data gathered daily from the web, with more than two billion entities (people, places and things) and tens of billions of associated facts and relationships between those entities.
“It’s really inspiring when you look at all the data we have stored in our repository and see how rich is it,” says Silviu Cucerzan, a member of the InstaFact team and a principal researcher at Microsoft.
“The challenge is: How do we put all this information into the hands of users in a nice, simple way, so they don’t have to keep switching tools and rely on results from web search engines?”
While the user interface is simple – a single click triggers the tool — building the system was complicated. The tool has to correctly infer what a user wants, search terabytes of data, retrieve accurate facts, auto-complete documents and do it all very quickly.
“We had a lot of challenges initially,” says software engineer Srivatsava Daruru. The knowledge graph was built for Bing’s search engine, and to leverage it for Office users the team had to combine many technologies for natural language processing and graph traversal.
“In the end, it was very beautiful when we got all of them working together,” says Daruru.
The team also wrestled with speed. To be useful, the tool must be fast, but a large spreadsheet with many entities and items to auto-fill presented a big challenge.
“We can get the entities fast with our entity-linking technology, but if there are hundreds of entities, we have to do the graph traversal in parallel for hundreds of things, and results must be back in one second for this to be compelling,” says Cucerzan.
“It has to be InstaFact. Half-Hour-Fact is not interesting to anybody,” he adds, laughing with the rest of the team.
Everyone on the team works in Microsoft’s Bing Satori group, but none of them had worked together before the Hackathon. None had experience with a //oneweek Hackathon. But during the frenzied, three-day event, the team spent long days and nights in a conference room to create a tool that was just a kernel of an idea just six weeks earlier.
They built and tested different machine learning models to see which ones gave the most accurate results. If you write “Rehnquist, Thomas and Kennedy” without context, the tool knows you’re referring to three Supreme Court justices and that “Thomas,” an extremely common first and last name, is Clarence Thomas. But because algorithms and data can be wrong, the tool mitigates for ambiguity by sometimes also giving users a list of facts to choose from.
On deadline day, with a few hours remaining, the team raced to debug the tool, smooth out the experience in Office and shoot a demo video required for competition — a task none of them had done before.
“By midnight, we were in panic mode,” says Cucerzan, who worked in Microsoft Research before joining the Satori group. “It was one of those crazy, beautiful days.”
The team now envisions InstaFact empowering many people and organizations, from students to journalists to businesses that need accurate statistics and research. The technology would get smarter over time as more people use it. And it could include an update button that auto-refreshes data on subjects like stock prices and population figures. The team hopes to ship the technology in the near future.
Software engineer Deepak Zambre says the tool creates a “distraction-free” environment that helps people focus on their work, a capability highly praised by Hackathon judge Mary Czerwinski.
“InstaFact ‘automagically’ brings the power of Bing’s knowledge graph into Microsoft productivity applications like Word and Excel, saving the user precious time and cognitive load, since they don’t have to app-switch to get the answer or information they need,” she says.
For Team InstaFact, software may be the expertise, but the endeavor to improve and do more is what inspired them.
Software engineer Rohit Paravastu recalled a project from grad school, in which he worked on a program for journalists to auto-fact-check reports during the 2012 presidential election.
“I had just moved to the U.S. and did not know anything about American politics, so I would search for the data and write it in my report,” says Paravastu. “This kind of tool would have been really helpful.”
For software engineer Rajeev Kumar, who also didn’t grow up in the U.S., struggling to write English made him more compassionate and want to help others write better and achieve their goals.
“We can help people by not just correcting spelling, but by giving them facts and new types of knowledge and making everyone’s lives easier,” he says.
Lead image: Team InstaFact is (from left) Srivatsava Daruru, Rohit Paravastu, Silviu Cucerzan, Rajeev Kumar and Deepak Zambre. (Photo by Scott Eklund/Red Box Pictures).