Kinect Makes Learning Playful with Help from Sesame Street and National Geographic

Editor’s note – Sept. 18, 2012 – “Kinect Sesame Street TV” and “Kinect Nat Geo TV” are now available. See the blog post and b-roll video for broadcast journalists.

REDMOND, Wash. – Oct. 18, 2011 – For more than four decades, children have dreamed about stepping into the world of Sesame Street and interacting with their favorite characters. Starting this spring, Elmo, Grover and the rest of the gang will invite them to do just that.

The upcoming “Kinect Sesame Street TV” experience, announced today at an event in New York and going on sale in the spring, will bring true interactivity to the classic children’s TV show. Viewers will be invited to jump into Sesame Street and play. Microsoft’s motion-sensing camera will let children interact through voice and gesture with Sesame Street characters who will respond based on the child’s actions. If Cookie Monster says “stand up and clap your hands with me,” he’ll recognize whether the child is playing along and interact accordingly. Or if Grover asks a child to throw a coconut, he’ll look to where the child “threw” it based on how hard they heave.

That interactive experience is something the people at Sesame Street have dreamed about for years, said Miles Ludwig, managing director of content innovation and Sesame Workshop.

“No one had ever delivered on the full promise of interactive TV – to bring an experience that delivers that same level of engagement and the same focus on character and narrative but also provides meaningful opportunities for kids to be a part of the story,” Ludwig said. “Now kids will be able to have adventures with their Sesame Street friends and be a part of the adventure in a way that’s never been possible before. We’re very excited about this partnership with Microsoft that will capitalize on new technologies to bring altogether new kinds of educational and of course entertaining experiences to families.”



Through “Kinect Sesame Street TV” children will be able to learn right alongside their favorite characters, like this young girl who is counting coconuts with Grover.

“Kinect Sesame Street TV” is one of many new interactive experiences Kinect will start delivering for kids in the coming months, said Dave McCarthy, general manager for Kids and Lifestyle Entertainment at Microsoft Studios.

In New York today Microsoft introduced its Kinect “playful learning” experiences, which aim to use the power of Kinect to make learning with games and TV fun and, for the first time, truly interactive. As the Sesame Street TV example shows, these experiences aim to break down the wall between TV and viewer – and hopefully the wall between entertainment and education, McCarthy said.

Microsoft today also unveiled other new games and experiences that will let children use their bodies and voices to play, including the following that will be released this holiday season:

  • “Double Fine Happy Action Theater,” which gets kids off the couch and having fun without rules, menus or instructions.

  • A new Pixar title, codenamed “Rush,” a new game in Microsoft’s collaboration with Disney.

With the announcements, Microsoft hopes to establish Kinect for Xbox 360 as the best gaming and entertainment device for kids, McCarthy said.

“There’s a variety of new experiences coming to Kinect this holiday and beyond for kids,” he said. “Whether it’s getting them up and having fun, getting them curious and exploring the world, or bringing them a whole new category of playful learning where we’re changing the way they experience TV and making it interactive and enriching, Kinect offers the best experiences for kids and families.”

Making TV Interactive

Several Kinect “playful learning” experiences will revolve around television, which for better or worse is the medium kids spend the most time with, McCarthy said. Microsoft is collaborating with Sesame Street and National Geographic, two partners that have delivered quality children’s TV programing for decades. From counting with Grover to exploring the natural world with National Geographic, the collaboration introduces true interactivity to children’s TV programming for the first time, McCarthy said.



One of the “Kinect Nat Geo TV” experiences will feature Nat Geo WILD host Casey Anderson and his pal, Brutus.

“Kids’ TV has always been a one-way street,” he said. “A character asks you to jump up and clap your hands, but it doesn’t really know if you’re doing it and can’t respond. With Kinect, we saw a way to make TV truly interactive – to make it a two-way conversation.”

For example, “Kinect Nat Geo TV” will bring NatGeo WILD to life starting this spring. In New York Microsoft previewed an episode of ”Kinect NatGeo TV” where Kinect will scan the viewer’s living room and transforms it into an animal habitat, complete with grass growing on the couch. Children will then be invited to forage for food as a bear, which they just learned about through the show. They’ll see an image of themselves onscreen with claws and a bear head and will roam around the living room – er, the rugged outdoors – trying to eat as many moths as they can.

The goal behind the “Kinect Nat Geo TV” experience is to transport kids and their families around the world to expand their knowledge of geography and the environment while inspiring a sense of wonder and excitement, said Brad Dancer, senior vice president of digital media and research at the National Geographic Channel.

“Because our programming is so information-rich we can only show so much, but ‘Kinect Nat Geo TV’ offers the ability to go further than what the television show can do,” he said. “It lets us go further into animal behavior and what it means, and give people the choice to learn more. It uses the best parts of interactivity and television content production to offer something new.”

And as mentioned with “Kinect Sesame Street TV,” children will see their image appear alongside favorite characters such as Big Bird, Cookie Monster, Grover, and of course, Elmo.

Microsoft is also collaborating with Sesame Workshop on a literacy and learning experience for kids in the four to seven year-old age range. Codenamed “Project Columbia” aims to bring story books to life and to kindle a love of reading. Through Kinect, children and families will be able to step into the pages of their favorite books and interact with characters, play with words and letters, and build their own stories. “Our goal here is not only a really entertaining experience that brings books to life like never before but also to deliver a tool to help aid the battle against childhood literacy problems,” McCarthy said.

Disney, Other Games Coming to Kinect

In June, Microsoft announced its first game in collaboration with Disney, “Kinect: Disneyland Adventures,” which goes on sale starting Nov. 15. Today, as noted above, a new game in collaboration with Disney was announced – codenamed “Rush” – that will bring Pixar characters and stories with Kinect in the spring of 2012. In the game, players will team up with characters from “The Incredibles,” “Ratatouille,” “Up,” “Toy Story 3” and “Cars 2” to race against time and solve fun challenges.

Other new kid-friendly Kinect for Xbox 360 games that are already available or coming soon include:

  • “Sesame Street: Once Upon a Monster” from Tim Shaffer and the Double Fine team. To play, kids open a living story book full of monster friends. This game is available today.

  • “Kinectimals: Now with Bears,” where children can play with bear cubs who respond to voice commands like “jump” and “rollover.” This game is available today.

  • “Kinect Fun Labs” lets kids animate their favorite toy, build an avatar, and finger paint in 3D. Fun Labs is available today.

  • “Kinect Sports: Season Two,” which features six new sports including American football, golf, baseball, winter skiing, darts and tennis. It goes on sale Oct. 25.

  • “Dance Central 2” builds on the most popular dance game on Kinect, which is very popular among young people. It goes on sale Oct. 25.

  • “Kinect Star Wars” from Lucas Arts, which is coming in early 2012.

Expanding Children’s Imaginations and Love of Learning



The Kinect “playful learning” experiences aim to break down the wall between TV and viewer – and hopefully the wall between entertainment and education, said Dave McCarthy, general manager for Kids and Lifestyle Entertainment at Microsoft Studios.

A key part of today’s announcements revolved around the new Kinect “playful learning” initiative, a program where Microsoft is creating engaging experiences that have genuinely enriching content for children and their families, said Dr. Alex Games, director of curriculum development for Microsoft’s Interactive Entertainment Business (IEB). Through “playful learning,” Microsoft aims to combine its entertainment and game design expertise with the latest education research to expand children’s imaginations and their love of learning.

“Playful learning is a concept that starts with play and acknowledges that through play, a lot of incredibly deep learning can happen,” Games said. “We are in the process of developing playful learning experiences that we believe can transform the way we think about play, its value for our day-to-day learning, and how we use it in our everyday lives.”

For 30 years, research in learning science and developmental psychology has shown evidence that play is a fundamental form of learning and important to a child’s development, Games said. To validate its current and future efforts with playful learning, Microsoft has launched a partnership with the Games for Learning Institute. The organization is a group of universities including New York University, Columbia Teachers’ College, and City University of New York that explore the potential video games have for learning. Microsoft will continue to work closely with the Games for Learning Institute and other academic institutions as it creates new playful learning experiences for kids, Games said. For example, Microsoft has commissioned several studies that examine how Kinect might impact children’s reading experiences.

“We want to find out what happens when a technology as powerful and engaging as Kinect is brought into the process of learning how to read in ways that are more active, more playful, and hopefully more engaging with kids,” he said.

Games said that today’s announcements only hint at the potential playful learning experiences that will be coming to Kinect for Xbox 360. Learning to use a controller has often been a barrier to playing video games for kids (and some adults). Kinect removes that barrier and makes the games and experiences on its platforms accessible to young learners. (Parents, of course, will be the ultimate arbiter of access, Games said, explaining that Xbox LIVE offers parents controls to make sure their children only get age-specific, curated content.)

“As we look into the future and Kinect evolves and technology gets better and more immersive, so will the playful learning experiences that we will be able to create,” he said. “With the current round of ‘playful learning’ experiences, we’ve put a strong step forward in creating the first gen of entertainment titles that effectively blend the state of the art research on learning with state of art game design.”