An Inside Look at the Months-long Process of Getting Windows XP Ready for Release to Manufacturing



Bill Gates, chairman and chief software architect of Microsoft, and Jim Allchin, group vice president of Windows, hold master code disks of Windows XP as the operating system is released to manufacturing on Aug. 24. Click on photo for high-resolution version.

REDMOND, Wash., Aug. 24, 2001 — If you think retailers, PC makers and customers are glad that Microsoft. finished development of its new Windows XP operating system today and began the process of manufacturing and distributing the software, consider Jack Mayo. He had to eat mealworms to get to this point.

Mayo, project manager for Windows XP, doesn’t make a habit of eating bugs. But two months ago, Mayo and other top managers on the Windows XP development team knew they needed to keep morale high if they were going to meet todays
“release to manufacturing”
(RTM) date — the final milestone in ensuring that millions of retail packages and pre-loaded PCs would be ready as promised for the products official debut, set for October 25.

“These last weeks had been pretty rough on our development staff,”
recalls Mayo.
“People were coming in at all hours, test teams were working in shifts around the clock. We were bringing massive amounts of coffee and soda into the offices. People would sack out on sofas for a bit and then get back to work. One guy just stayed here and slept on the floor, pulling an old event banner over his head.”

With the key features of Windows XP settled long before, the primary task facing the development team in these final weeks leading up to RTM was to eliminate the remaining
“bugs.”
This meant anything from adding prettier icons to cleaning up misaligned fonts to fixing bits of software code that didnt run properly. Thats when the top managers hit upon what seemed like a good idea at the time. If they wanted the development staff to get rid of bugs, they should be willing to do the same.

So a deal was reached: For each 250 software bugs that the developers could eliminate in a day, a team leader would eat a real bug — a mealworm. If the developers could eliminate 750 bugs in three days, then all four team leaders — Mayo and Senior Vice President Brian Valentine, Project Director Iain McDonald, and Director of Test Darren Muir — would get
“mealy mouthed.” “The developers came through for us; all of us ended up eating our meal worms,”
says Mayo.
“But only Brian chewed.”

Hitchcock-Like Suspense

Thats one of the more flamboyant episodes in a saga that has been filled with Hitchcock-like suspense right up until today. For 10 months, up to 85 Microsoft developers and program managers, test teams, and leads

representing thousands of their colleagues have participated in daily
“war team”
meetings to review status, identify problems, and set milestones for progress. Over that time, Microsoft has issued several beta versions and release candidates — preliminary versions of Windows XP that have given users and the industry an opportunity to try out the software, to get acquainted with its new features, and to offer feedback.

Microsoft has a variety of programs in place to solicit feedback from customers , ranging from computer manufacturers, software and hardware vendors, and large multinational corporations to small businesses and end users. Microsoft also conducts focus groups, extensive usability testing, and primary and secondary research.

For corporate users of Windows XP, some of the most valuable feedback came from 29
“Joint Development Program (JDP) partners”
— a select group of large and small companies that tested and deployed Windows XP early and participated in a nearly constant exchange of information with Microsoft, including weekly updates, recurring conference calls, quarterly meetings in Redmond and frequent product planning sessions. Participants got a leg up on deploying Windows XP in advance of its official release; Microsoft got invaluable advice from customers using Windows XP in real settings.

JDP participant Enterasys Networks began testing Windows XP with 200 of its corporate users in April; two weeks ago it began widespread deployment to its 2,700 PCs and laptops. The advance experience with Windows XP enabled the company to flag a compatibility issue with one of its internal applications and to identify
“wish list”
features they thought would make the user experience even better.

“The extent of Microsofts willingness to make Windows XP development an iterative process with us impressed me,”
says Len Couture, chief information officer for Enterasys.
“Their openness in receiving our feedback leads me to believe that this product is really focused on improving the user experience.”

As development proceeded, the focus at Microsoft was on trying to eliminate the last few
“bugs”
from the software. The team identified problems and assigned the appropriate developers. Those developers would write new code and forward it for inclusion in the software. Microsofts computers would automatically incorporate those changes and create a new
“build”
of Windows XP. Daily, that build would go to the
“Build Verification and Test”
Lab for review and identification of new bugs caused by the changes.

Where only weeks earlier the team had faced hundreds of bugs a day, only a handful showed up by the beginning of last week. By August 17, only three bugs remained. There would be only one more build — set for August 18 — before RTM.



Bill Gates, chairman and chief software architect of Microsoft, with a master code disk of Windows XP. Click on photo for high-resolution version.

“The three bugs we saw at the end of last week were relatively minor, and actually taking a night off at that point had its appeal — but we wanted to make the right decision for our customers,”
says Mayo. The team eliminated those three bugs and the final build was finished at 3 a.m. Saturday morning. The Windows XP developers had won the battle.

Burning 128 Disks Every 12 Minutes

But theyd not yet won the war. All day Saturday was consumed with burning the final build of Windows XP onto test media and distributing it to Microsofts internal testers to make sure that the software would install and run properly. Testers also checked that each of the many Windows XP versions — upgrades and full-product versions for retailers, pre-installed versions for PC makers, limited-time evaluation copies, and so on –had the appropriate software. To enable this testing, Microsoft deployed four CD-burners, each capable of burning 32 disks at a time, in 12-minute passes. In all, 1,000 CDs were created and distributed to testers that day.

At the same time, the developers began creating the international versions that also had to be ready by launch date. Fortunately, all of the binary code — Windows XPs actual operating instructions — remains the same for all language versions; the only element that changes from version to version is the
“central language resource,”
which tells the software what language to use to express those instructions.

By 9 a.m. on Saturday — just six hours after completing the final build — the German-language version was ready for testing. Given the longer lengths of many German words and phrases, compared to English, it was important to make sure that fonts and typefaces appeared correctly. Japanese, Korean and Chinese versions were also created for testing in Redmond and in their respective countries. Additional language versions — for Arabic and Hebrew, among others — were created for -testing in the countries in which theyd be used.

Mastering the Art of


Master Media

On Sunday morning, August 19, the team began burning the
“master media”
that would be used to duplicate all the various versions of Windows XP. While Saturdays tests had ensured that the product worked properly, the testing that began Sunday morning was designed to ensure that the media itself — the CD-ROM disks — contained no flaws. More than 300 pieces of master media were created for the English-language versions of Windows XP alone. Throughout the week, each piece of master media has been put through a battery of tests to ensure that upgrades, clean installations and continuing usage all function properly.

In addition, each piece has been subject to dual sweeps with three anti-virus checks, the first to check the CD image itself, and the second to check
“exploded”
or uncompressed versions of all the compressed files on the disk.

Twice-daily
“war team”
meetings this week — at 9:30 a.m. and 2 p.m. — monitored the final media testing process. By Thursday afternoon, all testing was done. Microsoft testers logged onto an internal Web site to sign off on 4,300 separate aspects of the software. With those sign-offs complete, everyone went home for the first full nights sleep that many of them had enjoyed in quite a while.

Windows XP Goes Out into the World

Early today, representatives from six major computer manufacturers planning to offer PCs with Windows XP pre-installed came to the Redmond campus. They received sets of master disks to take back to their manufacturing plants to begin the process of imaging them onto PCs.

Microsofts developers today finally reached the end of their long road to RTM. Jack Mayo, like most of them, wont have long to bask in the glow of their accomplishment. On Monday, Mayo begins work on the next version of Microsofts Windows .NET Server product.