With Code Jumper, experts look to jumpstart computer science interest for kids who are blind

WORCESTER, United Kingdom — It’s a warm day in early June and a group of students have gathered in the IT classroom at New College Worcester to show off their coding skills.

But instead of poking at tablet screens or typing into laptops, these students are taking out brightly colored plastic pods, connecting them together with thick white wires and then adjusting the pod’s buttons and knobs. These physical components will be used to create computer programs that can tell stories, make music and even crack jokes.

The students at New College Worcester are all blind or with low vision, and they are part of a group of students across the UK who have spent the previous school year beta testing Project Torino, a research project that led to the development of a new product called Code Jumper. It’s a physical programming language that is designed to be inclusive of children with all ranges of vision.

“What I like about Project Torino is that you can actually touch, physically, the program,” said Victoria, 14 and a student at the school, which serves about 80 students.

Microsoft has announced plans to transfer the research and technology behind Code Jumper to the American Printing House for the Blind, a nonprofit based in Louisville, Kentucky, that creates and distributes products and services for people who are blind or with low vision. Over the next five years, APH plans to offer Code Jumper and related curriculum to students throughout the world, with a target audience of students who are 7 to 11 years old.

The nonprofit’s leaders say the goal isn’t just to introduce kids to coding – it’s also to give them the underlying skills that can lead to a career in computer science.

“This is an opportunity for thousands of people to have meaningful and well-paying jobs,” said Larry Skutchan, director of technology and product research for APH.

The impetus for Code Jumper began about four years ago, when Cecily Morrison, a Microsoft researcher and computer scientist, began exploring technology options for her son, Ronan, who was born blind. Morrison was surprised to discover that many of the technologies available to Ronan and other children who are blind were far clunkier and more out-of-date than the smartphones, tablets and other technology that most kids today use, because few organizations had the resources to modernize them.

“A lot of it really felt like technology of the past,” she said.

What’s more, she found, the most popular path to introducing young children to coding, usually called block coding, wasn’t very accessible because it couldn’t easily be read with assistive technology such as a screen reader or magnifier.

“It became really clear that, for a 7- or 8-year-old, it was going to be really hard to use assistive technology to code,” she said. “We realized we really need something physical, something that would excite the hands.”

Click here to load media

None of this would come as a surprise to the students at New College Worcester. As they assembled their programs, some recalled computer science classrooms in which they had been told to do touch typing – which is a method of typing by muscle memory instead of looking at the keyboard – while the other kids in their classes used block coding to write basic computer programs. Others shared the frustrations of trying to learn more complex coding skills like Python or JavaScript without the grounding in more simplified coding systems many other students were getting to use.

With Code Jumper, they said, they were immediately able to experiment and build programs.

“I just felt very independent, and I liked that,” said Daniel, who at 11 years old already knows he wants to pursue a career in computer science. “It kind of made me inspired to do more coding.”

On this day, Daniel was collaborating with another student, Rico. The 12-year-old’s favorite subjects are IT and science, and he said that in his previous school he only got to do touch typing. With Code Jumper, he was able to write an actual program for the first time.

“To just do coding, it was a fun experience,” he said.

As the kids worked on their programs, Jonathan Fogg, head of computing and IT at New College Worcester, walked around the classroom, helping them with things like debugging. Fogg also watched with evident delight as they ran their programs for him. He said Project Torino was unlike anything he’d been able to provide the kids before now.

“There really isn’t an equivalent to this physical way of programming,” Fogg said.

The early access to basic coding skills is important, Fogg said, because many kids who are blind or low vision are drawn to careers in computer science. He thinks that’s partly because many of the skills kids with low vision develop to navigate the world make them good at the kind of computational thinking that’s helpful for a computer science career. And, he said, traditionally it has been a career that is more accessible to people who are blind or low vision, because of tools such as screen readers.

But at a young age, he said, he’s found that many kids are afraid to start playing around with a computer, especially if they know it’s an expensive, fragile machine.

“If they’re not confident, they won’t have a go at the computer because they’re afraid they’ll break it,” he said. “But once they’ve gotten over that barrier, then they’ve been successful. Project Torino reinforces that – they can’t break it and they can do all these really, really cool things.”

Magnets, blocks and lots of trial and error

It’s a couple of days later, and Theo is sitting in a small resource room at the school he attends, Kings College School, also in the UK. Theo, who is blind, has been part of the Code Jumper project for years – in fact, he was one of the first group of students who collaborated with Morrison and others to develop the system.

“I helped them choose what kind of buttons to use,” he said as his hands moved swiftly along the bright plastic pieces, assembling a program as he chatted easily about the computer science skills he’s developed over the years.

The bright plastic pods, oversized dials and thick cords the students are using today is a far cry from the original ideas Morrison and her collaborator, senior researcher Nicolas Villar, had when they first started thinking of a physical programming language.

Their original idea was to create a physical programming language that mimicked block coding, complete with actual blocks and magnets. It didn’t work at all. The kids either lined the blocks up in a row and didn’t do anything else, or they grew frustrated by the magnets falling off the table and getting lost.

So, Morrison, Villar and the other team members began meeting regularly with a small group of kids and getting their ideas. Based on the kids’ feedback and ideas, they switched to bigger plastic shapes that fit easily into kids’ hands, and they created surfaces the kids could rub or squeeze in order to recognize and interact with them.

With the new designs, the kids immediately began exploring ways to put them together and write short programs that made sounds.

Through the work with the young collaborators, Villar said he started to see the technology from the kids’ perspectives. For example, the kids with some vision benefited from bright, contrasting colors. They also found that kids like to work together, guiding each other’s hands, so they built the pods to be about the size of two kids’ hands.

“They were really helping us invent,” Villar said.

Once they had figured out what Morrison called “the kid way of engaging with things,” they set to work making sure that the system also would teach kids the basics of coding, such as how to create a sequence and what steps you need to take in order to debug.

The researchers also created detailed guidance that teachers without a computer science background could use to help kids develop coding skills, since they couldn’t expect all schools to have an IT specialist available to work with students who are blind or low vison.

For Villar, the impact of Code Jumper has extended far beyond just this project.

“It’s opened my eyes to different perspectives, different ways of focusing or seeing the world,” he said.

For Theo, the involvement with Project Torino has been life changing as well. He’s now doing more complex coding, including recently writing a hangman game in Python – something he says he couldn’t have done without the basics he learned using Code Jumper. Equally important to him, he’s made new friends with other kids in school who are also interested in coding.

“It’s been really good socially for us,” said Elin, his mother.

Big plans

In late 2017, Villar flew to Kentucky to show the leadership at American Printing House for the Blind a demonstration of the project. Craig Meador, APH’s president and a longtime educator, said he was immediately drawn to how intuitive and interesting the system would be to kids.

“If you put this in a classroom, not only is the blind student going to be using this but every student in the classroom is going to want a crack at this,” he said. “From a teacher’s perspective, that’s all you ever really want – something that’s inclusive.”

Skutchan, APH’s technology director, said the fact that it’s immediately accessible for kids with no prior computing experience is important. In the past, he said, kids who are blind or with low vision who wanted to get into coding had to first learn other computing skills, such as how to type and use a screen reader. That made it harder to get started at a young age – and potentially closed the door to a field that could have provided a career for them.

“I think coding is an equalizing field to be in,” he said.

Meador and Skutchan have grand plans for Code Jumper, including developing a curriculum and figuring out distribution and other levels of support. APH plans to release Code Jumper first in the US, UK, Canada and Australia, and then to distribute it throughout the world.

They say it’s the type of system they may have dreamed of – but would never have had the resources to create themselves.

“We’ve never really had anything that’s been able to give students so many different ways they can experiment and learn about coding,” he said, “and it translates so well into actual skills.”

Related:

• Using AI, people who are blind are able to find familiar faces in a room
• Code Jumper gives children who are blind the tools to code
• These tactile blocks teach blind kids to code
• Learn more about Code Jumper
• Learn more about Project Torino
• With Project Torino, Microsoft creates a physical programming language

Allison Linn is a senior writer and editor at Microsoft. Follow her on Twitter.

Top Image: From left, students Daniel and Rico and IT instructor Jonathan Fogg laugh about the program Daniel and Rico created during a beta test of the technology behind Code Jumper. Photo by Jonathan Banks.