Sous-Chef app at Hackathon helps you organize recipes, find new ones and cook with Cortana

Jane Koh’s mother loved to cook and collect recipes, making everything from delicious Korean barbecue to big Thanksgiving dinners.

When she passed away, Koh inherited the recipes, a treasure trove of cookbooks, handwritten notes, index cards and magazine clippings. Koh, who also loves to cook, has long wanted to find a way to use her mom’s recipes more easily with technology.

That was the genesis for the Sous-Chef app, a new Hackathon project that turns paper recipes into searchable text and keeps them with online favorites in a single, convenient, searchable place. The app also helps people find new recipes online and integrates Cortana, who can read instructions, set timers and convert measurements.

“The kitchen in general is really stuck in the last century. I don’t think there’s a whole lot of technology geared toward people in the kitchen,” says Koh, a Microsoft partner business evangelist in developer experience.

“Sous-Chef is for anyone who has paper recipes and bookmarked recipes strewn about the Internet that they want to get in digital form to use on a device while cooking, without getting kitchen goo on paper recipes or their devices.”

Based in New York, Koh is leading a diverse team this week at Hackathon, Microsoft’s annual innovation event, in which 12,000 employees are expected to participate worldwide. Sous-Chef team members work in the United States, Germany and England and include technical evangelists and software developers, plus a software engineer, a hardware engineer, a tech writer, a design researcher and a user-interface designer.

The team also includes Wee Hyong Tok, a senior program manager on the Information Management and Machine Learning team. Tok co-wrote a book on Azure Machine Learning released last year.

Sous-Chef uses Microsoft’s optical character recognition (OCR) software to turn images of paper recipes into searchable, editable text. It also uses Azure’s cloud-based search, analytics and machine learning tools to help users find online recipes based on such customizable factors as diet, cuisine and ingredients.

Sous-Chef can automatically suggest quick dinners on a weeknight, festive menus for holidays and recipes tied to what’s in season, helping people break out of cooking ruts and eat more healthily.

“I’ve been waiting for the opportunity to work on a kitchen technology solution for a dozen years,” says Amy Taricco, a product copy writer for Power BI, who was part of the initial team. “It’s time we can search all our recipes in any form, from anywhere and any device.”

Cortana can guide users in the kitchen by reading a recipe’s next steps, answering questions like “How long do you poach an egg?” and setting multiple timers.

“How awesome would it be if you could say, `Cortana, set a timer for 20 minutes for the rice. Cortana, set a two-hour timer for the turkey,’” says Koh. “Something like that would really be such a time-saver.”

For Valentina Grigoreanu, the app is a helpful way to keep and organize special occasion recipes. Every Thanksgiving, she asks her mother-in-law for her apple-cider-brined turkey recipe, which has to be faxed over. Every summer, she asks her mother for a family pickling recipe, which her mom digs up online and modifies to match the family’s recipe.

“There are certain recipes we might only use once a year – and lose them every time, in our big pile of recipes,” says Grigoreanu, a senior design researcher who works on improving product usability. She also sees potential to create personalized recipe books with different themes.

Daniel De Freitas Adiwardana, a software engineer on the Microsoft Dynamics CRM team, is also working on Sous-Chef, having been interested in speech recognition tools for years.

“Voice recognition technology has come a long way and will soon be in every Windows 10 device,” he says. “So I hope I can help apply it in a very valid, hands-free scenario, which is cooking!”

Adiwardana, who is working on Sous-Chef’s Cortana integration, started cooking basic meals after he moved from Brazil to the United States to work at Microsoft. Among his first dishes: Brazilian cheese bread and milk pudding from paper recipes his mom gave him.

Koh worked at a startup before joining Microsoft, where the passion and collaboration at Hackathon reminds her of her roots.

“I’m thrilled to be working with such a brilliant team of both technical and business-savvy folks,” she says. “This Hackathon shows that creativity is valued at Microsoft. Great ideas can come from any source, even from non-technical folks such as myself.”

She’s excited to share her mom’s love of cooking – and her traditional Thanksgiving stuffing and potato gratin – with her relatives.

“Hopefully we can pass along some of these special recipes to the next generation,” she says.

Lead photo: Jane Koh in the kitchen. Photo by Phil Maier, styling by Natalie Kocsis.