Can AI help save penguins?

Penguins inhabit one of the most secluded parts of the planet, yet human activity is threatening their existence. Warmer temperatures associated with climate change are melting the Antarctic ice faster than ever, eroding the grounds where penguins live, feed and breed, while commercial overfishing and incidents like oil spills are depleting their food supply.

A 2008 World Wildlife Fund study reveals that if the global average temperatures increase by just 2°C – a distinct possibility over the next 40 years – around 50 percent of emperor penguins and 75 percent of Adelie penguins could disappear.

Conservation efforts for penguins are easier said than done. Their secluded existence in the Antarctic region means there is very little data available, and manned missions are difficult, especially during the harsh winters. To solve the data challenge, projects like Oxford University’s Penguin Watch have set up time-lapse cameras to monitor penguin colonies and click a photo every hour over many years.

But this trove of photos creates a problem of its own – every camera generates hundreds of photographs, making it impossible to count penguins in every photo manually. Researchers require a better and faster way, or they might run out of time.

Can Artificial Intelligence (AI) help?

Enter Gramener, a data science consulting company. Gramener specializes in solving organizational challenges by creating AI and machine learning-based models for large enterprises, and converting these insights into easily consumable data stories.

The penguin counting problem inspired the AI Labs team at Gramener, led by co-founder Ganes Kesari and lead data scientist Soumya Ranjan Mohanty, both of whom are passionate about using technology for societal good.

In partnership with Microsoft, they started attacking the problem from multiple aspects. But first, they had to train the model to identify what a penguin looks like from different angles and perspectives, before the model could even attempt to count them.

Thankfully for them, there was enough crowdsourced, labeled data from the Penguin Watch Project that gave them a jumpstart to train the model to identify a penguin. Once the model could identify penguins, the next challenge was to teach it count them accurately.

“We faced multiple challenges. Almost 70 percent of images were unusable, as either the lighting wasn’t good or it was foggy, and penguins weren’t visible. Then there is the problem of penguins overlapping each other and penguins closer to the camera appearing larger compared to those far away. These are easy for the human eye to perceive but difficult for an algorithm,” Mohanty explains.

Working on Microsoft Azure platform, Mohanty and his colleagues used a Convolutional Neural Network model to come up with a solution that can identify and count penguins with a high degree of accuracy. The model can potentially help researchers speed up their studies around the status of penguin populations.

The team is now working on the classification, identification and counting of other species using similar deep learning techniques.

Building AI to save the planet

A long-time Microsoft partner headquartered in Hyderabad in India, Gramener is not new to leveraging AI for social good using Microsoft Azure. It was one of the earliest partners for Microsoft’s AI for Earth program announced in 2017.

“I believe that AI can help make the world a better place by accelerating biodiversity conservation and help solve the biggest environmental challenges we face today. When we came to know about Microsoft’s AI for Earth program over two years ago, we reached out to Microsoft as we wanted to find ways to partner and help with our expertise,” says Kesari.

While the program was still in its infancy, the teams from Gramener and Microsoft worked jointly to come up with quick projects to showcase what’s possible with AI and inspire those out there in the field. They started with a proof of concept for identifying flora and fauna species in a photograph.

“We worked more like an experimentation arm working with the team led by Lucas Joppa (Microsoft’s Chief Environmental Officer, and founder of AI for Earth). We built a model, using data available from iNaturalist, that could classify thousands of different species with 80 percent accuracy,” Kesari reveals.

Another proof of concept revolved around camera traps that are used for biodiversity studies in forests. The camera traps take multiple images whenever they detect motion, which leads to a large number of photos that had to be scanned manually.

“Most camera trap photos are blank as they don’t have any animal in the frame. Even in the frames that do, often the animal is too close to be identified or the photo is blurry,” says Mohanty, who also leads the AI for Earth partnership from Gramener.

The team came up with a two-step solution that first weeds out unusable images and then uses a deep learning model to classify images that have an animal in them. This solution too was converted by the Microsoft team into what is now the Camera Trap API that AI for Earth grantees or anyone can freely use.

“AI is critical to conservation because we simply don’t have time to wait for humans to annotate millions of images before we can answer wildlife population questions. For the same reason, we need to rapidly prototype AI applications for conservation, and it’s been fantastic to have Gramener on board as our ‘advanced development team’,” says Dan Morris, principal scientist and program director for Microsoft’s AI for Earth program.

Anticipating the needs of grantees, Gramener and Microsoft have also worked on creating other APIs, like the Land Cover Mapping API that leverages machine learning to provide high-resolution land cover information. These APIs are now part of the public technical resources available for AI for Earth grantees or anyone to use, to accelerate their projects without having to build the base model themselves.

Helping biodiversity conservation organizations with AI

Along with quick experimentation and helping create the APIs, Gramener is also now working with organizations that are looking for solutions for their existing biodiversity conservation projects.

“Our collaboration with the Microsoft AI for Earth team has evolved over the years, and we continue to be inspired in executing Lucas’s vision of doing social good with AI,” says Kesari. “When organizations approached Microsoft to help them with their conservation projects, we got involved by bringing our expertise on top of Microsoft’s platform and creating custom solutions for their specific needs.”

They worked with the Nisqually River Foundation in the United States, to help identify fish species in the river. The solution analyzes video footage from an underwater video camera that gets activated whenever any fish passes infrared sensors installed in a fish ladder at a dam, and identifies the presence of salmon. The solution, which utilizes Azure Data Science VMs, the Microsoft Visual Object Tagging Tool (VoTT), and the Microsoft Cognitive Toolkit (CNTK), has the potential to convert a task that used to take more than 100 hours by researchers to just 20 hours.

Thousands of miles away in Kenya, the team worked with the researchers to help them distinguish elephants from other livestock from aerial photographs. This solution can potentially help conservation scientists quickly identify elephants.

“Augmenting human intelligence by leveraging AI and storytelling has been the key theme of Gramener’s partnership with Microsoft. We are fortunate to be involved with Microsoft to create a positive impact in areas such as environmental sustainability and biodiversity conservation,” says Naveen Gattu, COO, Gramener.

When passion meets purpose

“Today, there is poor awareness about the massive environmental challenges and threat to biodiversity. With the democratization of data and technology, there is new hope for researchers and social organizations who are strapped for resources,” says Kesari. “I strongly believe that we can use AI to help solve these challenges and create awareness to bring about a revolution.”

The team at Gramener hopes that their work with data analytics and storytelling can help create more awareness among people and help organizations solve big challenges.

For Mohanty, who leads that AI for social good charter at Gramener, it comes down to one of the most basic human needs.

“When you work on such projects, you feel good about yourself. You get a good night’s sleep that day,” he quips.

Lead image credit: David Merron/Getty Images