Electronic pop duo Phantogram creates immersive music spectacle with Kinect

Elaborate video art at concerts was once the purview of bands like U2, headlining global stadium tours.

“Now, technology is democratizing access to immersive visuals and allowing musicians to turn their shows into really unique experiences,” explains creative technologist Blair Neal, who has worked with the electronic pop duo Phantogram to produce performances using Microsoft’s Kinect sensor.

Neal and Phantogram recently collaborated on a special one-off performance at the Hollywood Forever Cemetery. Neal used the openFrameworks open-source C++ toolkit to custom craft an application for the Kinect motion sensor camera. The technology enabled him to generate images onto the band’s bodies while simultaneously showing a real-time projection map of their moving silhouettes on the stage background.

Josh Carter, the guitarist and one of the vocalists for Phantogram says, “For us, it’s not just about the music. We’re very visual too and that helps make deeper connections with fans.”

Phantogram was formed in Greenwich, New York, in 2007 and consists of Carter and lead vocalist and keyboardist Sarah Barthel. The band’s name was inspired by an optical illusion in which two-dimensional images appear to be three-dimensional – so visuals are part of their DNA.

Neal is an artist and researcher who uses video and light as instruments to, in effect, plays along with bands. He and Phantogram share a philosophy on live shows in that they are always looking for special moments that deliver a sense of surprise and awe.

Barthel says, “We’d been working with Blair pretty much since we started as a band, but, for the Hollywood Forever show, we wanted to do something unique. The venue only does a few events per year and, of course, we’d never played a cemetery before.”

Neal continues, “We set out to make something magical with that ‘How exactly did they do that?’ experience for this show.”

Creative technologist Blair Neal uses light, video and Kinect to elevate live music experiences.

They decided on a concept wherein the band members would be the elements to create and drive the visuals themselves. Neal says, “I had seen Kinect used in similar work in modern dance shows and gallery pieces, but I wanted to bring that to a live concert situation.”

He chose to use Kinect because the depth image from the camera can isolate the musicians’ shapes and avoid other stage elements like microphone and keyboard stands. He made custom software using the openFrameworks creative coding environment, which Neal says, “is very supported by Microsoft and they are really leading the charge on it.” His code was set up to isolate the contours of their bodies and use these shapes to draw different generative geometries while Barthel and Carter moved around the stage. Kinect would feed data to Neal’s computer so he could control the visuals in real time.

Neal worked out the aesthetics, designed the physical projector, the Kinect wiring and coded the visual instrument. “The fact that I was able to do all of that work by myself in a few weeks is a testament to how these tools make it so quick and easy to get something like this up and running,” he says.

When the Hollywood Forever show started, Neal and the band realized there was a “happy accident” where the larger projection behind them, and the projection on Phantogram’s bodies, created a 3D effect from the perspective of the audience. Not a bad bit of serendipity for a band named after a phantom 3D visual trick.

“With bands like Phantogram, you have to represent texture, tone, mood, motion, cadence and layered patterns,” Neal says. “When we first used the Kinect visuals on the song ‘Don’t Move’ at the Hollywood Forever show, it was like ‘Wow…it’s working…this is really happening right now,’ and you could feel the audience respond with extra energy.”

Both Phantogram and Neal believe that interactive technology will play a larger role in music and art moving forward. The challenge is that many technologies define the interaction around their own limitations.

For Neal, the greatest appeal of Kinect is that it makes interactive technology more invisible, enabling artists to get closer to their intended concept and expression. He says, “Kinect strips a lot of that challenge away and just leaves the magical ‘How does this work?’ feeling that we aim for. Bands can now accomplish things that were impossible or crazy expensive even a few years ago.”

The show was a success with fans and the band alike. “The visuals made Hollywood Forever Cemetery my favorite show ever,” says Barthel. “It was absolutely epic.” With so much potential and so many options, Phantogram and Neal are already looking forward to their next big projects together.

Lead image: Sarah Barthel and Josh Carter from the electronic pop duo Phantogram on tour in Dallas.