REDMOND, Wash., April 19, 1999 — Jeanne Sheldon, a test manager at Microsoft, led the complex effort to test all of Microsoft’s Office applications to ensure they can handle the upcoming date transition to the Year 2000.
Testing the products was no small feat. The team started with the latest version of Office first, testing Word, Excel, Access, PowerPoint, Outlook and all the software applications that make up Microsoft’s business productivity suite. They tested all 33 languages in which each of these products is available. And they tested each language of each product for a variety of date scenarios using several operating systems and configurations. The team then repeated this process for all versions of Office dating back to October 1993.
“It’s very time consuming,”
Sheldon said.
“You have to understand exactly what you’re testing because the behavior could vary depending on the version of the operating system or what underlying shared code may have been updated since the initial product release. Often the same tests had to be run many times with different configurations to narrow things down.”
The Office suite is among the core products Microsoft tested for the millennium bug. For the past three years, hundreds of people have worked thousands of hours to test Microsoft products to be ready for the year 2000. Testing is a key component of Microsoft’s strategy for helping customers prepare for the year 2000 computer problem. During the past few years, Microsoft has tested over 2,000 products and languages, of which 93 percent are compliant.
So which products does Microsoft test? And what does it mean by compliant? Before it began testing, Microsoft developed a list of products released since 1980, and made a judgment call as to which products it would test. The company decided to test products based on several factors: the size of the installed base; whether the product was used in an organization’s core infrastructure; the age of the product; the exposure to a customer due to an error; and customer and partner input.
The age of the product is a significant factor in the testing process. For example, a product such as DOS 4, which was compiled on an IBM 286 processor that is not even manufactured or supported any longer, is difficult to test because the environment has to be recreated. Such barriers make testing a challenging and complex process.
While most of the products Microsoft tested were developed in 1995 or later, the company purposely chose not to use dates as a cut-off point, because some pre-1995 products are still used by a substantial percentage of customers.
“Some companies have literally drawn a line that said 1998 forward,”
said Mark Light, Year 2000 product manager at Microsoft.
“We chose not to do that. If we felt there was a large installed base or that the product would have a business impact, we went ahead and tested it.”
In addition, the company adopted a general policy of testing all the languages in which a product is available.
“It would be easy to think that just because I tested the English version, I could extrapolate that to all the other languages, but we don’t do that,”
Light said.
”
If there’s a Thai version, we go test the Thai version. And the reason is that, on average, the European languages have about 5 percent unique code, and the Far Eastern languages have about 20 percent unique code. That’s a significant amount of code that’s unique.
“While we’ve tried to define year 2000 compliance in a way we feel is appropriate, it’s important to note that it’s really only the customer’s definition that’s meaningful to them,”
said Light.
“That’s why we have a policy of full, open and total disclosure of information.”
For Microsoft to label a product as compliant, it must meet many criteria. A major aspect of the Year 2000 problem is that many products store dates as two digits rather than four. As a result, when the year rolls over to 2000 (00), some computers will interpret the date as 1900 instead of 2000 and not execute calculations properly. To be considered compliant, Microsoft products must store and calculate dates consistent with a four-digit format.
In addition, if the product uses a two-digit shortcut for a year (00 instead of 2000) then the product must recognize the year as consistent with a four-digit format. In the case of 00, for example, the product must recognize the two-digit shortcut as 2000 instead of 1900. Also, for Microsoft to recognize a product as compliant, it must function properly through the year 2035.
“We didn’t just want to get customers across the transition to the year 2000,”
Light explained.
“We wanted customers to feel confident that the product was going to continue to work well into the 21st century.”
Another technical issue stems from the fact that many computer systems and applications do not recognize the year 2000 as a special leap year that happens once every 400 years. Many software applications recognize every fourth year as a leap year. But some don’t account for a more complicated leap-year rule, namely that every 100 th year is not a leap year–even though it’s divisible by 4–unless it’s also divisible by 400. As a result, some systems will cause dates following February 29, 2000 to be offset incorrectly by one day. To be considered compliant, Microsoft products must execute leap years correctly.
In addition, issues may arise from the fact that many older computer programs save memory by using date fields to signify other meanings. For example, 9/9/99 may have several special date meanings, depending on the programmer that wrote the code. In some applications, the use of the special date meant
“save this data item forever”
or
“remove this data item automatically after 30 days”
. These dates may have unforeseen effects, depending on how the date field has been used throughout the years. To be labeled compliant, Microsoft products must not use dates as special values.
To date, 93 percent of the products Microsoft has tested are year 2000 compliant, meaning they meet the company’s test criteria as is or with an available update, and any issues that are found will not compromise the core features or stability of the product. Of the 7 percent of products that are not compliant, Microsoft in some cases recommends that customers upgrade to a newer product free of charge. In many cases, the products are old and used by a very low percentage of the customer installed base.
Microsoft takes a three-step approach to testing, Light said. First, testers inspect the actual computer source code used to create a product to understand how it handles dates and evaluates whether it’s behaving the way it should. They then take the product and observe how it behaves as it moves across 20 to 30 key dates such as 9/9/1999, 12/31/1999, 1/1/2000 and 12/31/2000.
“And so you put in 11:58 on 12/31/1999, and you watch it cross over to the year 2000, and you say, ‘Does everything still work?'”
he explained.
Finally, Microsoft conducts custom tests on its products to ensure it will continue to handle the types of numbers for which the program was designed.
“For example, mortgage tables look forward, and birthday tables look backward,”
Light said.
“So each product group was given the rights to create their own unique tests they felt appropriate for the product.”
To communicate the results of its tests to customers, Microsoft posts information about each product on its
“Year 2000 Resource Center”
Web site located at http://www.microsoft.com/technet/year2k. In addition, customers can subscribe to Microsoft’s Year 2000 Resource CD, which provides updated information about Microsoft products on a quarterly basis. Those without Internet access can also obtain year 2000 resource information by calling Microsoft’s toll free information line at (888)MSFT-Y2K.
While it has completed the vast majority of its testing, Microsoft still has about 200 products and languages remaining on its
“to test”
list. In addition, the company will re-test products as new versions are released.
“We’re 95 percent of the way there,”
Light said.
“But we’ll continue to test any time a product evolves, and we’re pretty hard core about it. If it has a new service pack, we test it all again. If it has a new version, we test it all again. We are committed to continually testing products to ensure that customers are updated on the latest year 2000 findings.”
Because the testing is ongoing, Light said, computer users should continue to check the product status on the Microsoft Web site–as well as those of other companies–to make sure the evaluations have not changed.
“You can’t just go to a company’s Web site one time and make a static decision that you’re ready for the year 2000 and then never look again,”
he said.
“It’s very important to monitor progress of your infrastructure over time, because things can change. Not that they do very often, but it’s possible.”
Just as important, customers should take what is called an
“end-to-end”
approach to testing their computer environments,
”
Light said. The year 2000 status of Microsoft products is just one piece of the equation. In addition to monitoring the test results of Microsoft operating systems and software applications, customers should evaluate the hardware and software they purchased from other companies. They also need to examine any documents or spreadsheets they created using two-digit rather than four-digit years. And they should evaluate custom software applications they downloaded from the Web or received from programmers other than a software company.
“The Year 2000 computer challenge is an industry-wide issue that potentially affects any computer chip, computer system or software program that keeps track of dates,”
Light said.
“As a result, customers should take a comprehensive approach to evaluating their computer environments. Even if they think they’re compliant, they ought to check it periodically because new information and updates may have become available.”
