Skip to Main Content

Argonaut: Robot incorporates Windows 10 and the cloud

Frederick Fourie has always been interested in the life robotic – especially the rise of the devices amongst DIY hobbyists. And now the senior data scientist, who works in the Internet of Things group at Microsoft, has come up with a //oneweek Hackathon project focused on a motorized robot built on a Raspberry Pi board using Windows 10.

And Windows 10, he says, opens up all kinds of possibilities.

“The amount of storage space and processing needed to create a sense of awareness is too much for these little devices, so we’d do that up in the cloud,” says Fourie, who’s using the Windows Insider Program to get access to Windows 10 builds before the official launch on July 29. Windows 10 Core builds include hardware drivers for boards like the Raspberry Pi 2, so users don’t have to get bogged down in complex driver code. “We’d feed instructions to it. We have to figure out how to use the cloud to empower these little devices. That’s the idea, so we’ll see how far we get.”

Generally, hobbyist robotics projects feature boards such as Arduino and Raspberry Pi. These boards don’t have enough processing power to do both real-time sensor processing and higher-level processing, such as sensor fusion and training of neural networks. This project will attempt to upload the robot’s sensor readings and periodically download updated models for doing feature detection and navigation. A continually evolving model in the cloud can make the robot even more accurate over time, despite its limited resources.

Also, those familiar with the Windows development ecosystem can hit the ground running faster to develop Internet of Things projects like these. And, Fourie says, Microsoft has already created and published toolkits for getting devices connected to the cloud, which helps makers focus on the details specific to their project. This also lowers the learning curve for new programmers and helps hackathon participants do more in less time.

Fourie has been at Microsoft since 2001, and has worked in Windows, in Bing Labs doing the initial software for maps, Flight Simulator, and in Automotive, working on Sync. When he worked with the robotics group, they looked at what robots could do in the home. He knows there’s a robotics community at Microsoft and hopes that through this project, he’s able to find people who are interested in the same things. And if anyone is struggling with robotics, he hopes this project can help give them what they need to get over their hurdle. Fourie was able to overcome his own hurdles in putting together the team for this project by bringing in partners with expertise in hardware and programming.

“Robots are amazing because they are essentially inanimate objects that are brought to life. Building your own robot is even cooler, because you can make it do whatever you want,” says Argonaut team member Jonathan Chiu, who’s in charge of getting the camera working with Windows 10 IoT Core. Also a first-timer with the //oneweek Hackathon, he’s a software engineer on the IoT Client Quality team working on the Windows 10 IoT Core. “I’ve only recently built an air hockey robot with a coworker, so I’m fairly new to the robot scene. This is part of the reason I’m excited about this project. I’m learning a lot about getting the software and hardware components to work together.”

Muhamad Muhamad Lotfy, a software engineer works with the team that brought up Windows 10 on Raspberry Pi 2, is working on motor control and motion programming. His favorite part of the project is using Windows 10 and Raspberry Pi to do robotic locomotion and kinematics.

“//oneweek greatly emphasizes innovation and hacking,” Lotfy says. “It gives employees a chance to dedicate a week to work on ideas of their own coupled with fun, motivation and prizes.”

It’s easier than ever to get started with robotics, Fourie says, thanks to kits like the one he’s using for the project — relatively inexpensive and pre-assembled with hardware so that experimenters can get right to the programming. It’s downright revolutionary, he says, how these kits bypass what used to be steep learning curves for electronics and programming skills.

Frederick Fourie presents Argonaut, the robot his team is using to demonstrate how Windows 10 and the cloud can help devices like this do more around the house. The cups and tape represent a simple home layout and colors and lines that the robot is programmed to recognize. (Photo by Scott Eklund/Red Box Pictures.)
Frederick Fourie presents Argonaut, the robot his team is using to demonstrate how Windows 10 and the cloud can help devices like this do more around the house. The cups and tape represent a simple home layout and colors and lines that the robot is programmed to recognize. (Photo by Scott Eklund/Red Box Pictures.)

“A lot of hardware drivers for Arduino don’t exist, so people have to get their hands dirty and create complex code, hunt down the drivers,” he says. “With Windows 10, a lot of the drivers will come in the box, which means less code to write.”

In the experiment the team is setting up to show the potential for this technology, the camera-equipped miniature robot (affectionately named Argonaut – “Greek for wanting to be a hero”) maneuvers around Fourie’s Bellevue office, inside of an area on the carpet bordered by tape and including colored paper cups and soda cans.

The team will teach Argonaut to pay attention to where the lines are, as well as recognize and respond to the colors. Each color can signify something. For instance, red could mean the kitchen or yellow could stand for the living room. Processing in the cloud through Windows 10 will help the little guy learn about its environment. When Fourie was with the robotics group, they ran into the problem of not having enough processing power in the device to do harder tasks such as facial recognition and navigation around the house.

This could eventually lead to robots that are able to identify different members of a household and anticipate when they’re most likely to Skype, or watch certain shows or even let Mom know Timmy’s not doing his homework when he’s supposed to.

That expansion could also lead to more experimentation from students, who can mix and match different components from robotics kits to learn about programming and electronics by solving concrete problems and testing algorithms.

For Microsoft employees, the //oneweek Hackathon and its rapid prototyping is like being able to go back to school again. For Fourie, the Hackathon will kick start a project he sees evolving over several years.

“A lot of people have good ideas while they’re sitting in their offices,” he says. “When I started, you had to march according to the project you were shipping. I think a part of the changing culture is acknowledging that people have different backgrounds and very interesting ideas, and if you can bring those into the work environment then you really have a chance to get back to a start-up atmosphere. It’s amazing how quickly you can get to where you want to be.”

Frederick Fourie holds the Pixy Pet robot educational project published by Adafruit that he’s named ‘Argonaut’ for the 2015 //oneweek Hackathon project. (Photo by Scott Eklund/Red Box Pictures.)