REDMOND, Wash., Sept. 30, 2004 — Being the first customer to test a new product is bound to spark uncertainty. But entrusting the welfare of a company’s core financial, human resources and supply chain management information to a database platform that has never before been tested in any other enterprise? That takes confidence.
For several months now, Microsoft has been doing exactly that: running its SAP R/3 enterprise resource planning software system on the second beta version of Microsoft SQL Server 2005, the company’s next-generation data management and analysis platform. Microsoft’s Enterprise Data Services group also has migrated its Microsoft FeedStore data warehouse system to SQL Server 2005 Beta 2 to support the needs of diverse applications within the company.
Mike Hatch , senior technology architect of the Enterprise Application Services-SAP group within Microsoft’s internal IT organization, concedes that testing new software on production systems like these can be risky. But he says the practice of “dog-fooding” Microsoft products — as in “eating your own dog food” — is a crucial and invaluable step in ensuring that the products are truly ready for customers prior to release.
Hatch, who is responsible for all software and hardware supporting Microsoft’s SAP R/3 infrastructure, has overseen the migration and testing of SAP R/3 on each release of SQL Server and the Microsoft Windows operating system during the past four years. He and David Campbell, general manager for SQL Server, detailed Microsoft’s upgrade and testing experiences with SQL Server 2005 Beta 2 version in a keynote presentation today at the 2004 PASS (Professional Association for SQL Server) Community Summit in Orlando, Florida — the largest user-run event of the year dedicated to users of SQL Server.
Hatch earlier sat down with PressPass to describe what his group has learned from testing SQL Server 2005 in a production environment and the value of this dog-fooding practice to Microsoft’s product development efforts.
PressPass: Why does Microsoft take the risk of upgrading crucial production systems such as SAP R/3 to beta versions of its software such as SQL Server 2005?
Hatch: One of the main priorities of our IT group is to be the product team’s first and best customer for new products. If we’re asking customers to trust that Microsoft products are reliable, secure and ready to handle real business demands, we’ve got to be willing to do the same thing ourselves. As a result, Microsoft IT systems are running on beta software approximately 35 percent of the time. This dog-fooding process is a crucial part of the product release cycle — in fact, the product can’t ship unless we’ve signed off on it, so essentially we are a “customer” even though we work for the same company. Testing these products while they are still in beta allows us to feed critical information about their performance back into the product groups as well as verify that a product’s features are ready for customers to use in production situations. This includes documenting bugs in the product, filing design change requests and participating in regular feedback sessions with the product groups.
It’s true that we assume a certain level of risk in being the first to do this production testing, since there’s always a small chance that some unpredictable results may affect our system. But by the time we upgrade systems like SAP R/3 and FeedStore, we’ve already built up a lot of confidence in testing the product. We’ve upgraded to many beta versions and run systems on them for months, if not years, and we’ve never had an instance of unplanned downtime or data corruption due to the products.
PressPass: What lessons and feedback have you gained from this latest migration to SQL Server 2005?
Hatch: There haven’t been any big surprises, which is largely due to the reliability of SQL Server 2005. The beta product ran on our production SAP R/3 system, with a database volume of 1.7 terabytes, for 33 days without needing to be shut down or showing any sign of unresponsiveness. The only reason we did a restart after 33 days was to carry out a disaster recovery exercise. Another reason the migration has been so smooth is that we’ve gone through this process many times before and we have great processes in place for managing this type of early adoption program. It starts with us sitting down with the SQL Server product group to learn what their key goals are, then rolling out the beta technology in a very disciplined way. We’ve communicated closely with SQL Server product group about the new features within this product, we’ve tested them thoroughly and we’ve been consistent about filing any bugs that we’ve discovered. What we’ve learned is that SQL Server 2005 works just as promised.
PressPass: Do you think that your ability to work closely with the product teams puts you at an advantage over other customers who may not have that option?
Hatch: Because we have the advantage of working so closely with the Microsoft product teams throughout this process, and because our internal IT group is in many respects the most demanding customer when it comes to assessing products’ performance, no one else can test our products the way we do. Our early adoption program is very disciplined and methodical; our agreement with the product groups is that they can’t ship the product until it can do what we agreed it should do. If there’s a problem in testing, we stop and go back to the product group to get it fixed before moving to the next stage. This process is designed to give Microsoft customers the advantage of knowing that we’ve already anticipated and resolved any issues with the products before they ever ship, so customers don’t have to deal with those issues.
PressPass: Which new features of SQL Server 2005 have proved the most beneficial during the SAP R/3 and FeedStore migrations?
Hatch: For the SAP system, two key features in particular are the database-mirroring components and the online indexing tools. Online indexing is a feature that allows us to reorganize data stored in SQL Server 2005 without taking the system offline, which is crucial for maintaining high availability and uptime with a system that’s as mission-critical as SAP R/3. Database-mirroring improves upon the SQL Server log shipping features we use today to maintain a standby database. With database-mirroring, we have a “hot” standby system that is highly synchronous and provides recoverability with no transaction loss for the SAP R/3 environment. Database-mirroring also gives us the ability to create a highly fault-tolerant database system that protects all of our business information in the event that we have to do a failover on our primary SAP environment. We have also found that database-mirroring and online indexing are both very simple to implement, which was an exciting piece of feedback to give to the product group.
For FeedStore, we are very impressed with the vastly improved ETL (extract, transform, load) capabilities of the SQL Server Integration Services tool, formerly known as Data Transformation Services (DTS). These new tools provide us with an orchestration workflow and dataflow palette that includes drag-and-drop components for defining data sources, data destination connectors, process flow, transformation components and SQL Server general maintenance functionality. We currently have thousands of lines of code to do this type of thing, and we should be able to reduce that significantly with the new SQL Server Integration Services tools. The new Service Broker, Native Web Services, CLR integration and replication enhancements also give us new feature tools out of the box for services and capability extensions in our data integration applications. SQL Server 2005’s table partitioning features should help improve the performance of FeedStore in working with very large tables, so we want to delve into that as well. The list just goes on. This is a very rich enterprise database applications platform.
PressPass: What important business benefits have you gained from running SQL Server 2005?
Hatch: The overall system reliability and availability that we’ve experienced is really the most valuable result for us. In terms of enterprise reliability and continuous availability, there’s nothing more important than the ability to maintain a high-performance production system and a “hot” standby system with database-mirroring. We’ve been able to continue running our 1.7-terabyte database platform, with more than 300,000 transactions per day, on the beta product, and achieve performance benchmarks that are within a few percentage points of what we had with our previous SQL Server 2000 platform. Some of our critical processes are running faster on SQL Server 2005, and the product group has not completed its performance tuning yet. By the time we upgrade our SAP system next year to the final version of SQL Server 2005, we’re confident that the performance will be significantly better. On the Enterprise Data Services side with FeedStore, we’ve already seen a 10- to 15-percent performance gain in our SAP ETL jobs.
The new SQL Server 2005 tool sets also offer us a significant number of improved manageability benefits in running large SAP systems with tens of thousands of objects in the database as well as in its various subsystems. I’m also excited about a number of new scripting tools that enable our system administrators to make modifications and script out changes more easily.
PressPass: Can customers expect the same benefits that Microsoft has experienced with its beta testing of SQL Server 2005?
Hatch: Thousands of enterprises use SAP R/3 to manage their critical business processes in much the same way that Microsoft does, and I’d venture to say that the demands of our SAP production environment put SQL Server 2005 through a quality assurance process that’s as rigorous as any you’ll find in any business. We really haven’t had to do anything different in upgrading our SAP system to the beta version of SQL Server 2005 than we would have done with an RTM (release-to-market) product. If we’re able to reliably run SQL Server 2005 on the mission-critical systems inside one of the world’s largest companies, and do it months before the scheduled release of this product, we’re convinced that our customers will have even better results. Improving the customer experience is the most important motivator for us to go through this process.
