Microsoft Hyperlapse makes first-person videos smooth and speedy
Allison Linn
Microsoft News Center Staff
Professional photographer Nick Didlick always wanted to make a great ski movie, but he had long been stymied by the need for two things: Tons of pricey camera gear, and plenty of time to shoot.
Not anymore.
A few weeks ago, on a brilliantly sunny early spring day, Didlick headed up to Canada’s Whistler Blackcomb Ski Resort armed with a handful of GoPros, a pair of willing athletes – and a plan for making an action-packed ski movie with the help of a new technology called Microsoft Hyperlapse.
By the end of the day, he had most of the footage he needed to create a short, time-lapse movie showing two athletes flying down the mountains of British Columbia, first on skis and then on bikes, before leaping into the water.
“I could quite easily make an entertaining ski movie in a day,” Didlick said.
Now, everyone else can give that kind of filmmaking a try, too.
On Thursday, Microsoft released Microsoft Hyperlapse, a new set of products that create smooth, stabilized time lapses from first-person videos.
For consumers, Microsoft Hyperlapse Mobile offers the chance to turn any long video – from a bumpy bicycle ride to a family stroll in the park — into a short, distilled version that you can easily share with friends and family. That technology, which is available for Windows Phone and in preview for some Android phone models, will come as a welcome relief to anyone who’s ever sat uncomfortably through a real-time video documenting those types of excursions.
For professional shooters and serious video enthusiasts, Microsoft Hyperlapse Pro can create a hyperlapse using a Windows computer from video shot on any camera or device – including aerial footage shot with drones.
Microsoft Hyperlapse Pro, which is available in free public preview, opens up possibilities that were previously difficult, if not impossible, for even serious filmmakers to achieve.
A third product, Hyperlapse for Azure Media Services, lets developers integrate hyperlapse options into websites and apps. That is currently available in a limited free public preview.
Creating more interesting videos
The idea for Microsoft Hyperlapse began in the mountains.
Johannes Kopf is an avid mountain climber, so when first-person action cameras such as GoPros hit the market, he naturally wanted to document his dramatic ascents.
But while the climbs themselves may have been exhilarating, the same couldn’t necessarily be said of the hours of footage he collected documenting each step.
“My mom would be the only person who would watch these videos,” he joked.
That’s when Kopf’s personal passion for climbing intersected with his professional passion for computer vision.
“I was really just looking for ways to make these videos more interesting,” said Kopf, a researcher with Microsoft Research’s Interactive Visual Media group.
In the summer of 2013, Kopf and a team of Microsoft researchers in the Computational Photography Group got to work – and quickly discovered that they’d stumbled into a pretty hard problem to solve.
“We got this idea of compressing time, then we discovered it’s not as easy as it sounds,” he said.
The biggest challenge was the footage itself. Unlike traditional time lapses, which are most often created using a static camera, video recorded with a first-person camera tends to be very shaky and uneven. That’s because the person shooting the video is usually walking, running, biking or otherwise in motion.
Simply speeding up the video – for example, just showing every 10th frame – just amplifies all those shakes and jerks.
“It’s not watchable,” said Richard Szeliski, a distinguished scientist at Microsoft Research who also worked on the project.
To make the video both speedy and smooth, the researchers came up with an algorithm that first creates an approximate 3D model of the landscape being filmed, and then identifies the dominant path that the camera took through that landscape. It then stitches together bits and pieces of different frames to create a smooth, stable hyperlapse that showcases the essence of the original video.
The technology also is designed to glide quickly over those times in which nothing much is happening, such as when you stop at a red light or pause for a drink of water.
Picking the right frames
For the version of Microsoft Hyperlapse designed for mobile devices, Neel Joshi, a researcher in Microsoft Research’s Graphics group, tried something a little different.
Rather than stitching together various pieces of frames, he created technology that looks for entire frames that have the most overlap with each other. Instead of just arbitrarily choosing every 10th frame, for example, the system might pick out the first, 12th, 18th and 29th frame.
This development, elements of which were also incorporated into the pro version, allows the hyperlapse to disregard a sudden jerk or jump, seeking out only the most compatible parts of the video. It’s being presented at this year’s highly regarded SIGGRAPH industry conference.
Joshi said the pro and consumer versions also rely on video stabilization technology that Microsoft researchers had previously developed, which further reduces the jerkiness often associated with first-person videos.
Vast possibilities
Although the idea for Microsoft Hyperlapse originated from an extreme sport, the researchers say they’ve had fun using it in much less extreme settings as well.
Joshi has tried attaching a GoPro to his dog, to create a hyperlapsed dog’s eye view of the world.
Szeliski has used it on vacation, while wandering through an outdoor market and taking a tour of a palace.
After Kopf and his colleagues released a paper about Microsoft Hyperlapse SIGGRAPH last year, Kopf said he was excited to hear from so many documentary filmmakers and other professionals who saw the potential to use Microsoft Hyperlapse to change filmmaking. But he’s equally excited to see what the rest of us can create with it.
“On the one hand, I would like to see a hyperlapse video going up Mt. Everest,” Kopf said, “but on the other hand I also want to see tons of amateurs use this for everyday stuff. I want to see people doing hyperlapses of their work commute.”
Related:
Microsoft Hyperlapse is available for download now.
Photos courtesy of Nick Didlick
