“I even remember the cake we had that time. It had an Android mascot on it, holding the Skype logo,” Kristel Saarepuu recalls with a smile from 12 years ago. Having worked as a software test engineer at Skype for a decade by then, it was time to celebrate her first major achievement – the launch of Skype for Android.
Kristel is now in charge of an automated lab for software testing at Microsoft Development Center Estonia. It’s a pretty impressive space: a relatively small area is used to house a variety of devices running different versions of the most popular operating systems in use today, from Linux to Windows, in addition to the mobile platforms Android, and iOS. In total, there are more than 700 devices in the lab, with 200 different iPhones alone.
The lab is needed for a very simple reason: to ensure that the software at Microsoft works correctly across all the operating systems in use, it is necessary to constantly check that the new programs are working properly. “As soon as a piece of code is ready, it has to be tested,” Kristel explains. “With the machines in our lab, developers can test code at different stages, whether it’s the beginning of a project or a finished user interface. This way, bugs that slip in can be fixed seamlessly.”
A global service area
Around 33,000 automated tests are carried out in the lab every day, and more than half a million tests a month. The lab in Tallinn serves a quarter of a hundred Microsoft teams worldwide. It would be inconceivable for each of them to have the necessary equipment with all the different operating systems and their versions, all properly set up, updated, maintained, stable and reliable. This is what Kristel’s team is responsible for.
In fact, Microsoft has labs in other parts of the world, but the lab in Tallinn is special in that it can also organize multi-part automated tests. “For example, if Microsoft Teams wants to test a multi-party call, the system automatically routes the test to the required machines, where the created program is installed. Then the testing can begin,” explains Kristel. The result is automatically sent to the developers, along with the video recording and log.
As the lab has users all over the world, and many of them are based in the US, the service must be available to them even when it is late at night in Estonia. Although the machines and equipment are carefully set up and stable, Kristel and her team keep an eye on the situation outside working hours. However, problems very rarely occur and are usually caused by something beyond the team’s control, such as a power cut.
Serving hundreds of people
Kristel’s team includes four other members. Considering that hundreds of people are served, this number is very small and therefore as many operations as possible are automated. “The machines must be reliable, accessible and safe, updated. We need to keep an eye on the state of both hardware and software at all times,” says Kristel. “With more than 700 devices, there are always some that need some kind of handiwork, whether it’s replacing a hard drive or servicing a mobile phone. The mobile phones in our lab are on charging mode, their battery can start to run down and we either have to replace the phone with a new one or have the battery replaced. That’s why they only last about a year,” she adds.
The weekly workflow, however, is based on a six-month plan, with the release dates of new products and operating system updates from different companies as key anchors. By the time a new device or operating system version comes into use, Microsoft’s products need to be tested, up and running.
The big players make their platforms available to everyone for this purpose: for example, the lab team is a member of Apple’s test program for initial or beta versions of its operating systems. In fact, anyone who pays the membership fee can join. Before a software update for a device is released, you can test the reliability of your products on it. However, it’s not so easy with new phones, which simply need to be pre-ordered and delivered as soon as possible.
Keep up to date with all the news
“I need to keep up to date with what’s happening in the world of technology, so that I have a constant overview of what’s going on and we can finish our preparations on time,” says the lab manager. “I’ve developed my own information channels for this, with automatic email and phone alerts on anything that might be relevant to the lab. Now the rhythm is in place, but it took a while to get used to it.” Keeping an eye on all this takes a lot of work and time, as the information flow is huge.
Kristel plays an important role for her team here: she sorts out what is important, how and when to react. “I try to protect my team from information overload and time overload so that they can focus on their work, and together we can ensure an infrastructure that is up to date and reliable,” says Kristel.
