Skip to Main Content

Extraordinary music and light installation uses Kinect to let people shape what they hear and see

Imagine not just listening to music or gazing at art, but becoming a part of it: Having the power to change what you hear and see, simply by moving and interacting with what’s around you.

A ground-breaking music and light installation opening in New York Thursday does just that. The Delqa installation uses Microsoft Kinect technology to give people walking through the New Museum’s NEW INC space the ability to manipulate and reshape the sounds and sights, closing the usual gap between artist and audience.

The fusion of music and technology draws visitors into the music of experimental pop artist Matthew Dear.

“I wanted to compose a sonic dreamscape, fully realizing that dreams are weird and not always black and white,” Dear says. “The music rests between peaceful and chaotic states, allowing the listener to come up with their own story while engaging with the piece.”

The project, which is open to the public Thursday through Sunday, highlights Microsoft’s broader ambition to give musicians, artists and tech-minded people new ways to work together to create revolutionary experiences and transform the future of music and art.

Inside Delqa, the transparent mesh walls curve and slope at unexpected angles. Wide-open spaces are interspersed with tapered corridors. Visitors are surrounded with light and music that change in sync with their movements and interactions.

The idea is to let people create their own experience and contribute to what others are seeing and hearing at the same time – a phenomenon made possible by the latest in interactive technology.

“The beautiful thing is that it’s impossible to predict how people will react to Delqa,” Dear says. “A project of this nature is full of unknowns, conjured up in workshops and sound studios. When we open the doors to Delqa for the first time, we’ll finally see where all of our wild ideas got us.”

Data from both the fabric inside the installation and the people moving through it are turned into “beautiful outputs that can drive sound and light,” explains Charlie Whitney, a media artist and developer in the project.

He says the materials were “specifically chosen to play to the strengths of Kinect,” a widely versatile technology used by music artists from Kanye West to Nine Inch Nails.

Matthew Dear (left) and Drew Seskunas discuss the project during a team planning session at the New Museum’s NEW INC incubator space. (Photo by Joe Mischo)
Matthew Dear (left) and Drew Seskunas discuss the project during a team planning session at the New Museum’s NEW INC incubator space. (Photo by Joe Mischo)

Dear worked closely with Whitney and a collective of other NEW INC members including animator Philip Sierzega, music interaction and spatial sound team Dave Rife and Gabe Liberti, music interaction developer Yotam Mann and environmental design team The Principals to determine how visitors’ movements could transform their environment.

The mesh fabrics are transparent to the Kinect’s color and infrared cameras but opaque to the depth camera, allowing the walls and the people within them to be tracked at the same time — as well as how they interact and move against one another in real time.

Steve Milton, founding partner of the agency that produced Delqa, Listen, says musicians and artists have been eager throughout history to broaden the ways they express themselves using new technology.

He says Delqa uses Kinect “to create new ways of experiencing music” by letting the audience help shape it.

Find out more about the installation on the Delqa website.

Lead Photo: Dave Rife, Yotam Mann, Steve Milton, Drew Seskunas and Gabe Liberti test the Delqa installation prototype at The Principals studio in Brooklyn. (Photo by Drew Reynolds)