Plastic bags. Plastic bottles. Plastic toys. Nearly 9 million tons of plastic debris wind up in the ocean every year. Environmental experts say the problem is serious enough that by the year 2050, the amount of plastics in the ocean could outweigh the fish in it if action isn’t taken sooner.
It was among the environmental concerns that gnawed at Drew Wilkinson. Two years ago, as a paralegal for Microsoft in Redmond, Washington, he had an idea for the company’s annual Global Hackathon, and contacted a nonprofit organization he greatly admired, The Ocean Cleanup, based in Rotterdam in the Netherlands. Maybe Microsoft could help?
Wilkinson called it a “shot-in-the-dark” email, the kind you send without knowing if it will land in the bulging inbox of an overloaded employee, or in a giant spam folder. And one that would travel 4,800 miles to another continent.
“I didn’t know anybody at The Ocean Cleanup,” Wilkinson says. “I just went on their website, and found their generic email address, and sent that email, not really expecting much. To my surprise, they responded.”
This was his email:
The communication was forwarded within The Ocean Cleanup offices to its geospatial analyst Robin de Vries. He received it like a Christmas gift in June.
“It felt that we could be helped by a network of experts,” De Vries remembers thinking. “This could be a great opportunity that only comes seldomly.”
I didn’t know anybody at The Ocean Cleanup … To my surprise, they responded.
The Ocean Cleanup is known worldwide for its innovative efforts to rid the ocean of plastics. It has also started focusing on eliminating plastics at major sources – rivers – before they reach the sea.
The organization built technology, unveiled last fall, that has been deployed in rivers in Indonesia and Malaysia, to remove plastics. But a key aspect was figuring out how to identify the waste that was collected – was it plastic or other material, such as sticks and leaves?
The Interceptor 002 in the Klang River in Malaysia. The Interceptor is headed next to the Dominican Republic and Vietnam. Photo credit: The Ocean Cleanup.
For two Microsoft global hackathons, in 2018 and 2019, Hackathon team members in Redmond and from around the world worked with The Ocean Cleanup to build a machine learning model to help quantify the amount of plastic pollution flowing down rivers en route to the ocean.
Subsequent models were then developed to replicate the process on cameras mounted to drones and ships crossing the ocean, and a blueprint for cloud computing infrastructure was created to help the project in the future.
Volunteers from Microsoft’s AI for Earth initiative also participated in the hackathon. AI for Earth is a Microsoft initiative that supports, and partners with, environmental groups and researchers to use artificial intelligence technology and advanced cloud software to solve environmental challenges.
AI for Earth sponsored the Hack for Sustainability Challenge at the hackathons in 2018 and 2019, with dozens of machine learning specialists, data scientists, software engineers, cloud architects, generalists and interns from around the company volunteering their time for The Ocean Cleanup.
Boxes outlined in red identify plastic items strewn in waterways that can be removed; boxes outlined in green show organic materials, meant to be left alone. This machine learning model was the primary output from the first Plastic Free Oceans project in 2018, proving it was possible to use AI to create a scalable solution to quantify and eventually remove plastic pollution from rivers. Photo credit: The Ocean Cleanup.
De Vries and another team member from The Ocean Cleanup, project engineer Kees van Oeveren, traveled from the Netherlands to participate.
“That decision turned out to be the fuse to a phase of explosive development,” De Vries says. “When we were in Seattle, we got introduced to so many aspects of the Microsoft organization that it became clear they could be a powerful ally in the quest for clean oceans.”
Before Wilkinson’s email arrived, Van Oeveren recalled the tedious work of labeling some images and doing the work by himself. “I hadn’t embarked yet on doing machine learning and image recognition, but I really wanted to,” he says, although he was not sure of the next steps.
The project was rudimentary, painstaking and time-consuming. “I remember mounting security cameras with power banks attached to them at rivers that would just simply store some images locally” on a computer, Van Oeveren says.
We labeled over 30,000 ocean photos with the help of Hackathon volunteers and their network.
Drew Wilkinson, third from right, with some of the 75 hackers who took part in the Plastic Free Oceans 2 project in 2019, and scientists from The Ocean Cleanup. Kari Lio, fifth from right, is another Microsoft employee who co-led the project with Wilkinson in 2018 and 2019.
It was a Tuesday when he shared a dataset of images with Microsoft Hackathon volunteers from the “Plastic Free Oceans” team to review. On Friday, in a video phone call meeting with the volunteers, he learned the dataset had already been put through a machine learning model, and bounding boxes had been created for the images, distinguishing what was plastic and what wasn’t.
“That was such a magical moment,” van Oeveren says. “I expected that even if you knew what you were doing, that it would be a project that would take weeks to set up right.”
“We labeled over 30,000 ocean photos with the help of Hackathon volunteers and their network” in the summer of 2019, De Vries says. “Some datasets were even finished before the Hackathon even started. Work is now in progress on the development of machine learning models for ocean photos.”
It allowed us to develop the vision that this is something we can do, not just for one river, but eventually for rivers across the globe.
Dan Morris, AI for Earth program director, says the most important result from the hackathon was that AI for Earth taught The Ocean Cleanup a lot about machine learning. “The real value was teaching them through interaction with data scientists and engineers at Microsoft,” he says.
This year, The Ocean Cleanup was named an AI for Earth grantee for its work.
“Using the AI for Earth grant, we’ve been able to set up and run the machine learning models,” De Vries says. “Having the resources at our fingertips has greatly accelerated the technical progress, by taking away practical concerns and letting us focus on the development.
“It allowed us to develop the vision that this is something we can do, not just for one river, but eventually for rivers across the globe.”
Robin de Vries, right, of The Ocean Cleanup works with a Microsoft Global Hackathon team member in 2019.
The Ocean Cleanup is highly admired, particularly in the Netherlands, where the organization has been a symbol of pride for years, even before they became more well-known internationally, says Harry van Geijn, a digital adviser for Microsoft in the Netherlands. Van Geijn is among the Microsoft staffers there who have volunteered to help The Ocean Cleanup when it comes to computer and related support.
While its staff is relatively small with around 100 employees, “they have this cause that they pursue with great tenacity and in an extremely professional way,” van Geijn says. So much so that “When I ask around for someone at Microsoft Netherlands to do something for The Ocean Cleanup, half the company raises their hand to say, ‘I want to volunteer for that.’”
Drew Wilkinson at the 2019 Microsoft Global Hackathon in Redmond, Washington.
Wilkinson, who grew up in the hot, dry climate of the Arizona desert, spent time at sea as a volunteer for the Sea Shepherd Conservation Society, a nonprofit, marine wildlife conservation organization.
In 2018 at Microsoft, he and another coworker started an employee group, Microsoft’s Worldwide Sustainability Community, which has grown to more than 3,000 members globally. The group focuses on ways employees can help the company be more environmentally sustainable. Wilkinson now is a community program manager for the Worldwide Communities Program, which includes the employee group he co-founded.
Wilkinson sees the issue of plastics in the ocean as a pretty solvable problem and is excited about the work that has been done, the work that he spurred with an email.
“I’m not a scientist, but it doesn’t take a lot of science to understand that our fate on the land is very much tied to the ocean,” he says. “The ocean is the planet’s life support system. Without a healthy ocean, we don’t stand a chance either.”
Top image: Some of the plastic and trash picked up onto the conveyor belt of The Ocean Cleanup’s Interceptor 002 on the Klang River in Malaysia. Photo credit: The Ocean Cleanup.