REDMOND, Wash., Oct. 12, 2000 — To commemorate the company’s 25 th anniversary, Microsoft this month published Inside Out , a book written by and for Microsoft employees that highlights the products, people and culture that transformed Bill Gates’ and Paul Allen’s vision for personal computing into reality. Featuring new essays by Gates and individual stories from hundreds of current and former Microsoft employees, the book provides personal anecdotes of triumph and failure, facts and trivia, and a look ahead to the company’s next 25 years.
Throughout the week, PressPass will publish excerpts and photographs from the book; in the stories below, test architect James Tierney and test manager Rachel Iwamoto explain what it takes to root out bugs and keep Microsoft’s software running smoothly.
Rachel Iwamoto
It’s true that sometimes developers aren’t delighted to see us. They work so hard, and then here we come, pulling their work apart. My test manager, Susan Higgs, used to work for another company. And she said that when they knew she was coming around with bug reports, some developers would run and hide in the men’s restroom. But that didn’t stop her. She would just walk in and chase ’em down. There are definitely times when you’re walking down the hall and you see developers ducking into doorways. They groan when they see you coming with all their bugs in a basket, but it’s just part of the job.
In order to be effective, testing has to happen at the same time as development. From day one, the developers start checking in code as they’re working on it. Then the builders take the code and compile it into programs that can be executed. Then the test team runs their different tests and automation tools on the build to find the bugs. The developers fix their bugs, the builders compile the new code, then the testers test. It’s a constant cycle of improvement.
We could tinker with code forever because there are always bugs in software. You can’t avoid it. It’s the nature of code, especially when you bring it all together from a team of developers where each one is working on a different piece of a tightly integrated product. That’s why it’s hard to know when you’re actually done working on a product and you can ship it. We’re constantly doing a balancing act–our customers want the product now, and we’re trying to get it right.
When we’re close to shipping, the developers stop adding new features and we move into an exclusively bug-fixing mode where all we do is find and solve the remaining few problems. Then the product leads do what we call
“triage.”
That’s where they go through the product and consider every remaining bug and set priorities for dealing with each one. We have to decide which bugs customers probably won’t hit, because we can’t fix them all, no matter how much we want to. There’s this old saying that goes,
“You have to shoot the tester to ship the product.”
I like breaking things. And complaining a lot about things that are broken. I always liked working on puzzles, and programming is a lot like trying to solve an incredibly complicated puzzle. You have to come up with a solution for complicated problems, whether it’s a software product for consumers or a testing program that will be used to perfect that software. The test programs we develop don’t always find all the bugs, though–sometimes they are found by just, you know, pounding on a keyboard. It’s not very scientific, but it works!
I spent my first decade after college as a software developer, and what I discovered was that it always took me longer than most people to get my programs to the testers because I couldn’t run them without finding bugs. By the time I gave them to the testers they couldn’t find any more bugs. So after sweating to get to code complete and finding my own bugs, I got to relax while my fellow developers were killing themselves to fix their bugs reported by testing. I can spot the good testers because good testers are the ones that can find bugs in my programs. If I have a bug and I can’t get it to reproduce, or it will only repro one time out of 10, or one time out of 50, then the developer in me assumes that it’s gone. But the tester in me never does.
A good tester will come up with some piece of test automation that keeps on hitting the code until he or she gets that bug to repro again. The best testers will keep on hammering on the code until they find all the bugs. A lot of testers actually have some kind of native ability to find bugs and they can’t shake it, even when they’re looking at their own code. So even though I’ve always loved writing software, that innate tendency to find bugs is the thing that makes me love testing.
I crash my cell phone at least once a month; I crash my microwave a few times a year. I always run into whatever bugs are out there. Software works much better for most people than it does for me. But software doesn’t always work that well for ordinary people either. When things go wrong with their software, most people think,
“I made a mistake. I need to do this differently so I don’t make that mistake next time.”
And me, I tend to blame the software. That’s what I always argue when I report my bugs to developers. I figure that if I did something that caused a program to crash, then someone else out there might do it, too. Some program managers and developers look at me and say,
“Are you nuts? Nobody would do that!”
Sometimes the tester and the developer have different ideas about how the product should be used or what the solution might be for a bug. But if we didn’t occasionally challenge each other’s assumptions, we wouldn’t be able to make good products.
When you think about it, writing test code offers the best of everything. You get to write a lot of code, which I love, but program managers don’t constantly change the product on you, and those pesky testers aren’t always bugging you about bugs. You just read the spec, analyze the problem, and then tackle it any way you want. I know several ways of doing business-as-usual testing, but I’m convinced that none of them are as effective as the underground methods you come up with when you’re really attacking the problem in an innovative way.
