What’s the link between Victorian whalers, Antarctic voyager Captain Robert Scott, freak snowfalls in 1939 and cutting-edge machine learning technology? The answer lies in Microsoft’s new project with NIWA, marrying historic weather records in New Zealand archives with breakthrough handwriting recognition tools, changing the future of climate science with data from the past…
The weather is a subject that never gets old – frequently changing, sometimes surprising, always a guaranteed ice-breaker. Yet few Kiwis know the story of a week their weather made history.
In July 1939, pictures of trams struggling through snowdrifts made front pages across the country, as many Kiwis experienced the first white winter of their lives. In Clevedon, Auckland, astonished locals were throwing the only snowballs they’d seen outside a Christmas card, while the summit of Mt Eden lay five centimetres deep. Masterton’s town clock froze under the weight of snow clinging to its hands. And flurries fell at the lighthouse at New Zealand’s northern tip – the equivalent of snow in Los Angeles.
The archives of the National Institute of Water and Atmospheric Research (NIWA) are full of such fascinating glimpses into our past, including copies of weather observations from whaling ships’ logs and Captain Robert Scott’s Antarctic expedition. In a world-first project with Microsoft, NIWA is now pioneering the use of artificial intelligence to scan old weather observations and turn them into data that lets us evaluate how our climate is changing over time.
Time machines to the future
The more we know about past weather, the better we can accurately predict climate patterns today and into the future.
“Was 1939 the last gasp of conditions that were more common during the “Little Ice Age”, which ended in the 1800s? Or the first glimpse of the extremes of climate change thanks to the Industrial Revolution?” asks Drew Lorrey, Principal Scientist, Climate and Environmental Applications at NIWA.
“Ships’ logs are like time machines, and we’re now using their legacy to help ours.”
Historic weather data is a tremendously valuable resource that could show the world’s leaders how our climate is changing, and help them prepare for what’s next.
“We’ve had snow in Northland in the recent past, in 2011, but having more detail from the past helps us characterise these extreme weather events better within the long-term trends. Are they a one in 80-year event, do they just occur at random, can we expect to see these happening with more frequency and why, in a warming climate, did we get snow in Northland?
“None of the New Zealand data for ‘The Week It Snowed Everywhere’ is included in the global weather reconstruction. By adding all this information, and much more like it, we can make the whole model more accurate and improve understanding about how storms have developed across the globe.”
Cloud science with cloud computing (and AI)
NIWA and Microsoft are working together on a world-first trial project using data from that frostbitten winter of 1939, determined to train machine learning tools to accurately transcribe the handwriting in old logbooks and draw relevant insights. The project has been awarded an AI for Earth grant by Microsoft to support local climate scientists in their quest.
Soon, NIWA hopes it will be able to scan an incredible 3,000 handwritten documents per day, upload them to the cloud, and generate searchable insights through Microsoft’s Cognitive Search application. This is already being used to assist in searching through the millions of files surrounding the JFK assassination.
“Old data is the new data,” says Patrick Quesnel, Senior Cloud and AI Business Group Lead at Microsoft New Zealand. “That’s what excites me about this. We’re finding better ways to preserve and digitise old data reaching back centuries, which in turn can help us with the future. This is data which is basically forgotten unless you can find a way to scan, store, sort and search, which is exactly what Azure cloud technology enables.”
There’s a special significance to the timing too.
Drew says: “This year is the eightieth anniversary of The Week It Snowed Everywhere, so it’s especially fitting we’re doing this now. We’re hoping to have all the climate data scanned by the end of the year, and quality control completed with usable data by the end of the next quarter.
“We’re using the records from 1939 as a discrete data set to test the quality of the automated handwriting recognition processes and improve data rescue procedures that will help solve known issues. The end result could be not only better global weather modelling, but much better methods of handwriting recognition that have so many other applications.”
An army of citizen scientists
The problem for climate scientists like Drew is the sheer volume of old data lying unassessed and vulnerable in old warehouses and archives. Weather records – some dating back to the mid-1800s – were meticulously kept in logbooks by sailors and whalers, with entries made several times a day that record information such as temperature, barometric pressure, wind direction and comments about cloud cover, snow drifts or rainfall.
Then there were the land-based observers. Drew recalls their names and biographies like a six-year-old rattles off Marvel superheroes. There’s Reverend Richard Davis in Northland, whose diaries of weather data helped early colonial administrators plan which crops to plant, and James Hector, founder in the 1860s of what later became the national meteorological service (and whose name was given to New Zealand’s rare dolphin).
NIWA has previously relied on an army of citizen scientists worldwide, all volunteers, to key the handwritten data into a computer database for analysis. The scale of the project is enormous, involving more than a million photographed weather observations from old logbooks being painstakingly reviewed and loaded by hand into the Southern Weather Discovery website. Many of them requiring some careful interpretation of the original writing. Not all record-takers had perfect copperplate penmanship, especially when the pitch and toss of conditions aboard ship was thrown into the mix.
NIWA’s collaboration partners at the National Oceanic and Atmospheric Administration in the US, the UK Met Office and universities across the globe face similar predicaments, with huge piles of old archived data ready to be recovered and analysed. Speeding up the data inputting and analysis with smart machine learning technology would enable the group, collectively known as the Atmospheric Circulation Reconstructions over the Earth (ACRE) initiative, to produce better global weather animations and a longer-term perspective of past weather.
Until now, however, computers haven’t caught up with humans when it comes to deciphering Great-Uncle Bertie’s spidery scratchings.
“Automated handwriting recognition is not a solved problem,” says Drew. “The algorithms used to determine what a symbol is – is that a 7 or a 1? – need to be accurate, and of course for that there needs to be sufficient training data of a high standard.”
But like the weather, things are changing fast.
“I’m chuffed that Microsoft has taken an interest in this,” says Drew. “NIWA has amazing environmental databases and part of our job is to ensure they are the best they can be. Microsoft has complementary skillsets and interests, bringing a huge wealth of experience in terms of cloud computing and the ability to create bespoke software solutions that enable us to improve the science we do for the benefit of New Zealanders.”
“Those who forget the past are condemned to repeat it”
And not just New Zealanders, either.
Microsoft’s Chief Environmental Officer, Lucas Joppa, sees a project that could quite literally be world-changing.
“This project will bring inanimate weather data to life in a way everyone can understand, something that’s more vital than ever in an age of such climate uncertainty.
“I believe technology has a huge role to play in shining a light on these type of issues and grantees such as NIWA are providing the solutions that we get really excited about.”
In the future, could artificial intelligence scan, interpret and cross-check mountains of quill-written documents to solve other mysteries?