advance08: The Future of Media

Remarks by Bill Gates, Chairman, Microsoft, and Satya Nadella, Senior Vice President, Search, Portal and Advertising Platform Group
Redmond, Wash.
May 21, 2008

ANNOUNCER: Ladies and gentlemen, please welcome Microsoft Chairman Bill Gates. (Applause.)

BILL GATES: Well, good morning, and thanks for coming to Microsoft. I get to talk about some of the ways that software is really going to change the game, not just in search and not even just in advertising, but change the game for so many of the activities that we engage in. You know, 10 years ago we didn’t think about software being related to how we organized our photos. We didn’t think about software being related to how we organized our purchases. And yet today, those are very much mainstream activities.

The miracle of software playing a central role, running on the PC, running on your phone, in your car, in the set-top box attached to your TV set, that’s going to change dramatically in the years ahead. So I’m here to share some of the ways that our research and development is going to redefine those experiences, and then in turn that will change the way we think about marketing and advertising and the user model of how they find information.

Now, why can we be so ambitious about this? What is it that makes this next period so special? Well, it’s the continued advance in both the hardware and software. And the hardware piece really is what got Microsoft started in the first place. It was Paul Allen showing me a 1971 article about putting a computer on a chip, and that that power would double every two years that made us realize that somebody should be doing the software.

And we can look at that over a period of time and almost go so far as to say that processing is free. More than a million times reduction in the cost of processing between the day that Microsoft was started and today. And in the years ahead, that will continue. There’s nothing slowing us down. We’re going to get many different processors, they will be in all the different types of devices, and they will get more powerful at the same time that they get cheaper.

So what kind of thing does this enable? Well, this is quantitative, you know, down to a very specific number in terms of the cost. The actual impact is more qualitative. So, for example, can computers see? Well, the answer to that to date has been, no, a computer — even if you give it a nice camera, it can’t analyze what it’s looking at. So, for example, if your TV set has a camera, it can’t tell who’s in front of that TV set and whether they’re paying attention or not, or whether they’re making some gesture.

In the years ahead, software, because of this processing power, will be capable of seeing. And so you can create a whole new way of interacting with the computer. In fact, we just brought into the marketplace a new product called Surface where we put a camera under a table-like device, and we can watch both your hand gestures on that and putting objects on it. And so that is a computer seeing what’s going on. In fact, Surface is being used in retail shops like AT&T phone shops where customers come in and interact and see different plans, they can put their current phone out, the phone they’re considering out, see what those features are, and it’s been fantastic.

So that idea of seeing will have that in many places — your desktop in your office, you’ll be able to have things displayed and just point and expand the information, your whiteboard will be an intelligent whiteboard and you can navigate through information there. So it’s pretty spectacular when you get what we call natural interface. Likewise, being able to talk to the computer, talk to your mobile phone and say what you want or have a tablet-like device that you can just take notes on and those notes can be recognized or searched, sent off to other people. That is the combination of incredible processing power together with software breakthroughs.

So where we’ve mostly been using the keyboard or the mouse to date, in the future, we’ll be using many ways of interacting because the computer will see, it’ll look at ink, it’ll understand speech, that’s a very, very big deal. It’s a big deal in the living room environment, in the house, it’s a big deal in the office, and it’s a big deal when people are out and about and the keyboard is completely impractical.

It’s also a big deal in terms of multiple people using the device. Personal computers have been very oriented towards the single person because you have that one keyboard that’s kind of set up with your one chair there, and you’ll see that whenever you take someone to your computer and say, hey, let’s navigate the photos, or let’s plan a trip together, let’s consider a purchase together, it doesn’t work very well for two people going back and forth and trying things out.

These new interfaces, including that Surface, having multiple people there, organizing the photos, picking what they want, it’s just incredibly natural because there’s plenty of room and the system is designed to take input from different people. So low-cost processing really is a driving factor.

Another factor that is easy to underestimate is low-cost storage. That’s gone down in this same kind of exponential way. There aren’t exponential improvements in any other domain of economic commerce — not in miles per gallon or food productivity or ditch digging, you know, any activity is subject to very sort of arithmetic type improvements, not exponential type improvements.

So with storage, if somebody had said back in the early days of Microsoft, okay, we’ll take all the meetings in the company and we’ll have them up online, we’ll listen to the voice track, we’ll make a transcript, people can come in and watch those whenever they want, they can even watch them at a slightly higher than normal speed so they don’t waste as much time. That would have been laughable because the cost of storing the video and the cost of having a network that could transmit that would be completely unreasonable. And so the video world grew up as sort of a tape-based, broadcast-type world. Well, now you’re all seeing that this has utterly changed, and Robbie Bach talked about how, with Media Room, that means that now we can think of video — even a mainstream video experience, TV — as not being something that you have to watch in real time and it’s only a finite number of channels.

The fact that a father can go to a sports event, take an HD camera, record that, upload it, and everybody from that school can watch that video, that’s a reality, and the cost is just negligible.

Inside companies, training programs are now done by putting the videos online and letting people interact to see that they’ve gathered those skills. And so all the training that doesn’t require group activity, a lot of which used to be very expensive, offsite, complex to schedule, is done just completely online.

Likewise, all the great college lectures are going online. It’s an interesting thing in terms of letting other universities take those great lectures, or individual students come and watch those. MIT started that and they’re seeing students from all over the world watch these things. I’ve surprised a few professors where I watched their course, sent them a little e-mail, tell them, you know, I loved their course, give them a few hints about how they might improve it. I mean, I don’t claim to be normal, but it’s just an example of when video is — storage is so inexpensive, this idea of putting things online just makes sense. You see storage coming together for things like maps and virtual reality. Wouldn’t have been possible without the cheap storage, cheap bandwidth, and cheap processing. But now we build software on top of that, and we get people used to these new experiences.

Moving these experiences to be digital, the digital revolution year by year this just gets more and more mainstream. I was in Korea two weeks ago, and Korea is the country with the highest broadband penetration. And so it’s been fascinating to track a number of things happening in Korea faster than other places. The way that online socialization works, you could have seen that there, absolutely, before it took place in the United States.

They’ve got a commitment to get rid of all the textbooks in their schools. They have many pilots running today — four years from now, their students will take tablet-like computers and simply use those to take notes and have their curriculum online, and none of the public schools will be using textbooks. They’re the first country who can say that because they know that when kids go home, they can be connected up, and it’s just expected, they’re willing to make the investment. And so this move towards a digital lifestyle, a move towards a digital work style, it’s happening in different countries at slightly different paces, but it is very much a global phenomenon.

You know, once one country gets in and does it, as the cost goes down, it becomes appropriate for more and more countries. And so we need to think about the activities that will really be different. You know, for example, we’ve had a huge boundary between the PC video experience and the TV video experience. That sports game I talked about that your kid was in that might be stored on some Web site, it doesn’t show up in your TV Guide, but it show up in your TV Guide. You know, my TV Guide should have all those physics lectures just show up automatically since that’s what would thrill me most of all. The system should know who I am and offer that experience not just on the PC, but on the TV as well. So that boundary is being eliminated.

The boundary between the PC and the phone, those are really very different experiences in terms of how you set things up, even you start a call on the phone, can you move it onto the PC and share a screen? Very, very difficult. You’re involved in manually thinking those things, but because of the power of the Internet, we can take those boundaries and get rid of them.

You know, you see we have the Microsoft Sync auto software here, we’re showing that. That’s been fascinating to us how quickly that took off. We knew the power in the car would get to be such that connecting up to the phone and using speech and those things would happen, but it’s actually happening even faster in terms of the strong user acceptance than we would have expected.

The TV change is one that’s very important because when you’re in this software-driven world, that’s where you can have a new show that you interact, a new show that you can skip if you’re not interested and see more about the things you care about. A great example of this is — of the divide we have today — if you take watching the Olympics, watching it on the TV versus the Web site for the Olympics.

The Web site is, in many ways, deeply superior because you can find the highlights, you can skip and go to the things that you care about. But then again, that’s usually just set up for one person on that screen. And so you’re giving up the living room benefit, that ten-foot viewing experience. And why can’t we have the best of both worlds? So the answer is that as the video distribution network moves to what AT&T has done in this country and people are doing in every developed country around the world, the answer is you can have the best of both of those things, you can see who is watching, you can have them give gestures and voice commands. You can have them socialize like you do with Xbox Live with other people while they’re watching TV, it’ll be utterly different.

The print-based reading, as we get these tablet-type form factors, you’ll read off the screen, and therefore the ads inside there can be targeted and interactive, just like the information itself is. So the digital revolution is at a very early stage. One of the ways that Microsoft is able to stay out in front of a lot of these things is by having a research group that’s trying out very difficult things. So speech recognition, you know, goes back 15 years that we’ve been working on it, but now it’s mainstream. Vision, over 10 years, but now it’s mainstream. Ink recognition, again, about 10 years. And these things — when they get usable enough and low-cost enough, then you’ll see a spike of adoption where something’s just been a few people messing around with it, it becomes a type of common sense. And having that research group that’s drawing on bright minds around the world, with locations in India and China and Europe and many locations in the United States, including a large group here, that’s what allows us to pull these things together.

So TV, will it change overnight? No, it won’t. But if we take the time period, like five to ten years, it’ll become utterly obvious that, of course, this type of flexibility is what consumers are interested in.

So let’s take a think about this in terms of finding information. What’s going on when you want to find information? In a sense, it’s not an end in and of itself. You know, if we take the extreme and just think of search, you know, is there anybody whose job is to search and they work all day and they get those links back, and then at the end of the day they go home and say, “Wow, whew, I did a lot of searching today. I should be paid a lot. Those links, you know, it’s so great when you get those links back on the page, that’s all I want to do is get links.”

Well, not really. You have some task in mind. You know, you’re writing a document and you want to look at, okay, who’s big in this industry, how have things changed. You’re trying to learn about something and you want the background on what is fertilizer, how does it work, why does it cost more money now than it did two years ago? You want to go and find those things. You’re collaborating with somebody, planning a trip, you want to look at different options about what’s out there.

The term that we first coined for this was “information at your fingertips.” And I think in some ways, that’s a very broad term that encompasses all these tasks, buying, reviewing, in business, making decisions about how to price a product, how to design a product. These rich activities are information rich. And so as you’re using software, communicating with somebody, creating a document, doing online transactions, you want information at your fingertips to enable that broader task.

You know, we’re making progress on this. The information on your PC, you have now this search capability. Your e-mail, you have great search capability. At the business level, finding the people who have expertise, finding the documents, knowing what’s going on, taking what has been very disparate, all these different quotes and things. We have an effort called SharePoint that’s really pulling that together and it’s getting the same type of adoption that Office itself has had for document creation in the office environment.

So broadly, we’re trying to enable information at your fingertips. And from 1990 onward, we’ve made a lot of progress. The Web is a huge part of this. A lot of the information you want to find is out there on the Web. And so Web search is an element, a piece of information at your fingertips. But you shouldn’t think of it over time as being isolated. You don’t want — if I’m in a document, creating a document and I want to do a search, everything I’ve written in that document should be helpful in terms of guiding the system into what I want to look up. You shouldn’t have to go off to a different window and just start from scratch and retype something.

Likewise, if you’re doing searching over time, the system should understand you, have a model of you that comes not just from searching, but from all the productivity and communications things you’re doing. So information at your fingertips is sort of the goal, and search is a very important piece of how that should be done in a very effective way. And so Microsoft, you know, believes in Web search, we think it’s a great thing. It’s fascinating for us, we have businesses like Media Room where we are completely alone, out in front, making the investment, changing the thing and there’s really no one to compare us to.

You know, then we have maybe some things that are kind of in the middle where — like video games or phone software where there’s lots of players and we’re very strong and part of what’s driving innovation. Then we have areas like search where we’re really an underdog. I have to say, it’s kind of fun to be an underdog. You know, it’s neat. (Laughter.)

This is one where we really have assigned a really amazing group of people to work on this. I’d say in terms of the depth of experience and breadth and really taking great talent around the company, we’ve done more on this to build a great team than on any effort that I can remember. You know, we’ve had a number of things, whether it was the challenge in word processing or the challenge in the browser where this type of assembling a team, seeing what other people are doing well, making sure that you can match that, and coming up with great things that we do that keep that area moving and innovative and dynamic, we have a lot of experience with that.

In this one, we have some neat things we’ve done. And we’re getting credit for some of the innovations even though we’re viewed as such an underdog. The resources involved here, the type of research, the server farms that today have hundreds of thousands of machines and the future will have millions of machines. The way that you design that, the way you do the air conditioning and electricity and rethink the way the system is designed and the software management, this is a fun, great software problem that actually has applicability in a fairly broad sense, the kind of deep, rich indexing, the scale, the machine learning algorithms that go against this. These are things that — the type of research people we have really can contribute to and have made a big difference.

If you look at where we are, you know, you see people look at what we’ve done in maps, people look at what we’ve done with some particular type of searches, they look at what we’ve done in news and they say, hey, these are things that are really better. They’re pushing things forward, and we’re on about an every-six-month cycle doing new and interesting releases. In fact, today sort of represents the culmination of our spring release. In fact, you know, this is where we’re announcing sort of the final, and a very interesting feature, that’s part of that spring release. But what you’ll see is every six months something pretty substantial, and we actually saved the research, we have things we’re working on today that we’ve been working on for a while that, until they’re released six months from now, some that are even further off than that because of the deep amount of software involved. So we’ve got a long-term commitment, a really neat strategy, neat group of people driving it.

I wanted to show — give you a little glimpse of some things that we think are kind of cool where if you’re willing to experiment with user interface, if you don’t view it as just completely — has to always stay the same, and you put some software technology in it, you can do some neat things.

So here we have Image Search. And I don’t have to think about pages. I just scroll down, I get more and more of these images. It also understands because it’s used image recognition, it can say you know, is it outdoor, indoor, is there a face in it. And so I can pick the images by different size . It also understands based on who this is who else I might be interested in. So Jack Nicholas might be interesting, I can shift over and get him.

So it’s taking this image path and trying to think what are you doing. And of course we can broaden it out a lot more as time goes on. If you look at that, Google it, you know, you’ve got the page issue, you have none of those ways to navigate, so it’s different. And we think what we’ve got is kind of neat there.

If you think about video, our lab — and this happened to be our lab in China — they worked where they can take the video and see the scene transitions and really understand how to give you a thumbnail view of that. So here I am in the search view, but if I just hold my cursor over it, you know, I can sit and watch the video. So I’ve got here on a page, over 30 different videos, but any one, if I just glimpse it, I can think, okay, is this the one that I’m interested in, and see it — and then of course I have, again, these related people where I’m thinking through — whenever somebody looks this up, what else might they be interested in? And then here we have the more traditional thing where you’re seeing fewer videos and the things aren’t related.

And so it’s great, when you’re an underdog, people are neat about seeing the advances you’re making giving you credit for those things, and we have a lot of those today. We keep adding to those things while changing how people think about search. So what are our big bets? You know, these are the bets that — this is what running a software company is all about. You make these bets and you stick with them for more than five years. You know, you bet on graphics interface, you bet on Internet browsing. Here with search, we believe that we can take it an embed it in a broader experience.

So we’re going to use all our assets and enable the information at your fingertips. So we’re going to take the task of buying, of organizing a trip with friends, of learning, and take the magic of software and say, okay, there’s more here than just giving you back a set of links. Search itself, which is a piece of that, is going to change. The way that search understands the different domains that you’re navigating, it will be a lot smarter. So if you just want to see different types of mortgage offers or ratings of different lawyers that you’re looking at, you won’t just go to the links, you will get that kind of information as something that is directly available to you.

And so there’ll be a lot of innovation and new business models will be part of the key of this, the way that the advertising revenue works, the way the relationship to the user works and is giving them some significant value. After all, in most contexts where you’re getting ads, you’re actually getting something in return. When you watch TV, they let you have that neat TV show, they actually give the TV show more minutes than they give the ads. In the search market today, when you get those ads coming up on your screen, in a sense, you don’t get anything back in return for that. And we think over time if you’ve got great search and you’re competing for those users, that the way that you benefit that user will be an element of it.

We’ve got a broad set of assets and we think long term, we love software, we start with the research type things. This kind of operation scale where you’ve got millions of servers, that’s actually something of immense value to the business market as well where we can take a lot of things people have had to do in their data centers, and when they have an overflow or just an app they want us to deal with the complexity of, we can run those in our data centers. So this big scale and the kind of redesign of the system around that plays to a great strength, and something that because of our role in business software, business computing, we are really quite unique in doing that.

We have the online properties, we’ve got the things that people do at work, and the PC has always been special in that it spans this work-home boundary. You know, Outlook e-mail. If you’re used to it at work, you’d kind of like to have those capabilities and scheduling things with your family and just have the information partitioned in the right way. So patient, long-term investment, the economic scale, and thinking globally about this because it’s a big market.

And when we think about ourselves as an underdog, you know, our share in some places, like in Europe today, is very small. It’s like new entry, breaking into the market in terms of the — if you take the specific element, which is the search-type share. And so how do you invest in terms of the breakthrough, the differentiation and the marketing there? Well, we’ve got the commitment to do that.

So how is search changing? In the last two or three years, not that dramatically. But if you look over a period of time, it has. And we think we’re entering to a period where there’ll be quite a bit of change. We went from directories, which was sort of a Yahoo! thing — there had been Alta Vista, but the keywords stuff wasn’t good enough. So people in the early stage were much more kind of directory oriented. Then the keywords stuff got a lot better, and that became the main way that people thought about it. The advertising model has also now moved onto this click-type orientation.

So this is something that works, but still see a very high percentage of the searches where they don’t get what they want. You see a lot of steps where they just want to get a simple answer to something. If you knew the user and had watched their pattern of behavior, you could do a lot better for them in terms of what they mean in taking them directly to the kind of information presentation they want. And so search can be dramatically better. Understanding the user intention can be a lot better. Showing a broader range of content because you know the domain that they’re asking about, that can be massively better. The way that an advertiser understands am I getting — not just clicks, but the kind of customers I want who are doing interesting things.

You know, why do so many of these searches go unanswered, even though we have these big indexes? Well, a pure keyword approach that is just sort of syntax, not semantics, that has limits no matter how much you build those things up, that’s not enough. You know, people want to get the things out there, they want refinement. They want, when a search doesn’t work, to know how they can go to that next step.

If you think of a specialized area like the travel search, you know, it’s ludicrous that the system doesn’t have more domain knowledge, your history in that activity, your preferences. And yet those domains, those are very, very big, and the ability of software to immediately tell what’s going on without expecting you to have to declare that is very straightforward.

You’re seeing a little bit of that where in the past you had to say, “Do I want images or links?” Now when you just use a main string query, you’ll get a little — some images if there are some that relate to that topic. So that’s trying to help the user out and not force them to go and click and just see the images over here and the rest of the things over there. But if you think about the domains that you do search in, it’s not just images and text, it’s very, very broad. And the kinds of things that you like to have mixed into those results varies by understanding that domain.

The context of user personalization, the context of taking search and using that for the display ads that come later, that kind of targeting, absolutely that’s going to be a huge thing because that’s where you get value and efficiency. If I had done on a search on topic X, you know, the next time a white space comes up, say I’m browsing a newspaper, the most valuable thing to stick in there is not related to that article, it’s related to that search that I did 15 minutes ago that said, hey, I’m interested in buying an expensive car, you know, more than the fact that I’m reading an article about Iraq, that should be very valuable to you.

So this boundary between what has been searched and what has been displayed, that boundary will get broken down somewhat as we’re driving for this more efficient approach, more targeted approach in where these things have gone. So you won’t just have the right rail, you’ll have other things that that user is experiencing in that same type of time frame that give you more flexibility, more real estate. So as we move into this next phase, it’s about moving from keywords to rich semantics. It’s about changing that user experience based on what they’ve done and what domain they’re in. And more on the economic model, this idea of many things moving to the payment relates to the action beyond the click. So paid engagement, and the user has a reason why they’re thinking, okay, I get some benefit if I’m choosing how I want to spend this time and look at these ads, I’m getting something in return for that.

And so here you can see this idea of thinking through different types of searches. You know, search has been incredibly horizontal and the breakthrough in terms of index size and all that. That was a great thing. We’re not moving back from that, we’ll have that as one of the enabling elements. But we’ll learn — just like in that image search, you saw a different set of things than you saw in video. Likewise, as you type in searches, we will look at that domain and have the ability to apply logic to that. We’ll have a platform so even third parties will be able to come in in domains they know very well and apply their logic. The user will often pick who they want to do that type of processing. So in, say, an area like sports, you may have different ways that that’s done that you indicate a preference for.

And so searches will not be as monolithic. You know, the types that are important, you’ve got entertainment, you’ve got commerce, reference, you’ve got just pure navigation where you’re literally using the search to get back to someplace that you’ve been before that you’ve heard about. So each of these deserve a very special way of thinking about the software. And of course, because the machine is very powerful, there’s plenty of room for that software.

So what’s the strategy? Well, broadly, best search results – a lot of progress being made there, and more that can be done. Think of the task, the broad task, not just providing those links and provide software around the experience to simplify that task, and then innovate in the business model.

Now, a particular focus for us where we see a lot of opportunity is in one of those types of search. We’re going to do a great job, we’re going to be the best across all those things and invest in all of them. But the one that you’ll see us particularly invest in a major way is a deep focus on commerce. It’s kind of interesting. Commerce represents about a third of all searchers, but a dominant share of the revenue that comes out of it.

And surprisingly, the level of satisfaction with these commerce searches is much lower than with references or entertainment searches. It’s more frustrating to somebody when they were just trying to learn about accessories for a camera that they get all the stuff about the cameras. They’re just trying to learn how to use it that they get the stuff about buying the camera. There are many different task types in commerce, and today the system doesn’t pull up the information from the different sites, let you see that in one place. It doesn’t let you distinguish the thing that it’s trying to do. So this is a very big part of the $20 billion search market, and in this particular area, the idea of simplifying the task and awarding both the consumers and advertisers for their engagement in a deeper way, we think this is — you know, there’s incredible possibilities here.

So there is a milestone here. I mentioned that one of the features of our spring release is just happening today, and that’s what we call Live Search cashback. It’s exciting. I think years from now you may look back and say, wow, you know, search started to get a fair bit more competitive and we can look back to that announcement. There’s a lot of elements every six months doing dramatic things, very long road that our long-term approach equips us for. But this is a pretty interesting thing, and I think is a great illustration of all the points I’ve been making about our strategy and how search can be made better for you.

This is giving you a reason why you should use a particular search, and that you get some extra value back. And so it’s not just the search engine capturing the value of your time, it’s the consumer and advertisers getting more of that economic value. And there’s a lot — you know, with Live Search cashback, we’re going to get a lot of feedback and a lot of ideas. We think what we’ve got is super neat here, but we can see taking this in some new directions and, in fact, we’re very interested in the input you have understanding your domains and your advertisers where we can go with this.

I thought I’d have Satya Nadella, who runs our search portal and advertising come up and give you a glimpse of why we’re so excited about Live Search cashback and this milestone. So come on out, Satya. (Applause.)

SATYA NADELLA: Good morning, everyone, and thank you, Bill. I appreciate the opportunity to spend some time digging in a little deeper on our commercial search strategy.

As Bill said, there are three things that we are focused on: delivering best results, simplifying tasks and innovating in the business model.

The way to think about our commercial search strategy is to look at the focus we have in these three areas, and really amplifying it through the lens of commercial intent. So, that’s really at the core of what we mean by our commercial search strategy.

And when we talk about commercial intent, we define it pretty broadly. We think about all the domains, if you will, that have high interest to end users, and high value to advertisers. So, things like travel, shopping, health, local, all fit in within the focus of commercial intent, and our focus of our commercial strategy.

The thing that I wanted to do is walk through each of these pillars that we just introduced, and give you a little bit of a feel on exactly what it is that we are doing from a technology perspective to enhance our search results, our search experience in the context of these commercial domains.

So in terms of delivering best results, there are two dimensions. Really, it comes down to having a very broad and deep selection or index, and then it comes down to having machine learning algorithms that can really work across the depth and breadth, and deliver the best results.

So if you sort of have to describe the breadth of the index, it’s definitely ever-growing. You have the Web results, you have images, you have video, you have all of the mapping data. You also have structured feeds, and especially in commercial domains you can get information about products and businesses. So there’s a spectrum, if you will, of information that needs to be available in terms of the breadth.

When it comes to depth, when it comes to just Web documents, we reached a big milestone last fall when we got to 20 billion documents. We are continuously growing that. It’s a capital intensive investment, so definitely something that we are going to stay the course in terms of growing that index, and also improving the quality of the index is a huge initiative for us. We have over three billion images in our index today. We have over 100 million videos. In fact, half of them have been annotated as smart motion thumbnails that Bill showed you. We have a petabyte’s worth of mapping data in our index so that we can use that as a geospatial back frame to a lot of interesting queries. We also have structured data that we’re ingesting in the case of product information, so we have approximately 30 million offers in our index today, 10 million SKUs behind that. We also have 14 million local business listings in the United States as part of our index.

So there is a real sophistication in terms of depth and breadth, but the interesting thing that we’re doing is not just collecting all of this, but really taking that index and making it much more semantically rich. And the way we do that is to understand some entities, like businesses, like products, and understand the relationship between these various entities. So, for example, when you’re looking at product, you want to get its price, you want to get its image, you may even want to get a video of a product that you may want to see before buying. So the ability to sort of take that index and grow it with semantic richness is a key piece.

And when it comes to driving relevance, our approach is a machine-learning approach. We want to be able to drive based on the query that gives the best results. We have a concept called the whole page relevance, so the first page you get back depending on your query has to have the best images, the best Web results, the best video, as well as the best ads. So we really want to have a ranking that spans all of these content sites, also spans the depth of the content we have. We also have domain-specific rankers, so when you go into product search, or you go into travel search, or you go into health, you have a much richer ranking capability. So the layering of these machine-learning algorithms to really help you get the best results is what we’re really doing. So that’s on delivering the best results.

The second piece is around simplifying tasks, and when it comes to simplifying tasks, it comes down to really understanding the user intent behind the query. No query is done in isolation, every query is done within the context of some tasks, and these tasks can be session long, or in some cases even multi-session. So our goal is to be able to really get to the intent and build experiences that match to those tasks, and that intent.

The second piece is for us to also provide intelligence by data mining in the context of the task. So, for example, if you’re shopping, one of the things that you may really want to know is what a user is saying about a particular product. So you’re using the search experience, or the search engine, to do research, and you would like to know what exactly is the entire Web saying about a particular product. So we mine the Web for comments on a product, and do semantic analysis on it, and provide something called the Opinion Index. So that’s intelligence for you as you’re researching products. Similarly, if you’re buying an airline ticket, you want to know whether the prices are going up or down, so that you can time your purchase. That’s, again, something that we mine a lot of different data to be able to predict the volatility of the airline prices. So intelligence tools, where we’re mining content, mining usage in the context of these commercial tasks is an additional element of our experience.

And lastly, we want to be able to take the richness of the experience, the intelligence that we have available, and extend it beyond just the PC experience. Definitely one scenario that I’ll show in a little bit is the mobile device, which we think in many cases can even be the place where you want that context, you want the task-centricity to manifest itself in pretty rich ways.

So that’s the first two pillars. Before I move on, let me just actually demonstrate a couple of these experiences. The first one I want to go through is the travel scenario. We acquired a company, a meta search travel company called Farecast a little over six weeks ago. We have now integrated that within Live Search as Live Search Farecast. What I’m showing you is the beta release of that. So of late many of us have been booking a lot of flights from Seattle to San Francisco in search of deals, and so what I’ll do is, I’ll go ahead and perhaps I’ll book a flight. In this case, I’ve typed in a query, flights from Seattle to SFO. I’ll go ahead and click on it, and instantly I get thing called an instant answer. And an instant answer here shows me, in fact, that prediction algorithm I was referencing, which is basically telling me that the fares are likely to go up, so this might be a good time to buy. This is one of the unique things that Farecast does where they mine a hundred million or so [pieces of] data on different flight information to be able to tell you whether the particular flight destinations that you’re looking at are good prices, or bad prices. In this case, it seems to be good prices from Seattle to San Francisco.

Let me go ahead and click on the link here. It takes me to something called the smart calendar, which allows me to pick the right date pairs in terms of when I want to leave and when I want to come back. So say I want to leave on Thursday, and come back on Saturday, I can get this $167; if I wanted to book for an entire crew, I can increase the number of people, and what-have-you. I can also, in fact, go ahead and look at what the prediction algorithm is telling me about the prices in the next 30 days, the peaks and valleys of, if you will, the airline fares, and that gives me the ability to pick the right dates to go travel.

So at this point I have picked the dates I want to travel. Let’s go ahead and click on the flight itself. So at this point what’s happening is Live Search Farecast is going out to all of the agencies, as well as airlines, and bringing back all of the flights that match the criteria of the dates I picked. So it’s a pretty rich search experience. You have in the middle all of the flight information. On the left-hand side you have the ability to refine this query, you can pick the airline, so the only airlines you want to see, you can pick price ranges, you can pick the number of stops, because depending on that, obviously, the prices change. So you have a lot of refinement tools that are domain-specific that we deliver on top of that.

We also have a time grid, so say I have all kinds of flexibility on when I leave, and when I want to come back, I can go ahead and say I want to leave early in the morning. I can pick that, and it automatically refines all of the prices for that early morning flight. So that’s a nice way for me to get the best deal, given the flexibility I have in timing. So at this point I’m ready to transact. So I can go ahead and click on any one of these flights, and then go to the site that it’s available from, and buy that ticket. So this is a great meta-search capability, on top of all of the flight information on the Web.

Having booked the flight, I may next want to go for hotels. So we have another service in beta, which is the hotel meta-search. In this case we have well over, I think, 100,000 hotels that we track, and that’s an ever-growing corpus for us. We have a similar prediction algorithm, even, for hotels. It’s called the Rate Key. So when you’re looking at hotels you want to know whether the rate that’s being currently offered is a good deal, an average deal, or not a deal, depending on historical pricing for that particular hotel.

I can also take a look at the map view of the hotel, since I may want to live close to the center of the city, or close the airport, or close to where I have a meeting. This map view is powered by Virtual Earth. So that’s another experience around hotels, which maps to the task of booking your hotel.

So those two examples were a quick way to look at how we can evolve search to first do meta-search in commercial categories, and really simplify the task of booking an airline ticket, or booking a hotel, all related to your travel.

So the next thing I wanted to show is another example around product search. So let’s say I’m in the market for a GPS system. One of the things that you use a search engine for is when you’ve not yet picked the particular product to buy, but you’re really in research mode. So I’m looking for a GPS system. I type that in as a query. I get back, again, an Instant Answer, which has got the most popular GPS systems, the images for it. Also I get back these buyer guides, just in case I wanted to look at the CNET buying guide, or buying guide for how to really choose a GPS system, I can go do that. So this is, again, the richness of the corpus of both the ingested, structured information, as well as the Web results.

So let’s say at this point I’m ready to go ahead and look at the GPS systems. I’m now going to Live Search product search capability, very similar to what I showed you with travel. In the middle you have all of the products, which I can scroll through, I can then refine on the left hand side by price, by brand, by category. So in this case, let’s say, something in the middle of the road. I go ahead and choose a price range. And then I can refine by user opinions. And this is the unique intelligence that we provide, because when you’re in research mode around products, we want to know what a user is saying about a particular product before I buy. So we were able to crawl the entire Web, look at all the comments people have left on GPS systems, ingested all of that, done sentiment analysis on it, and actually extracted features that people are writing about or commenting about.

So in this case people are talking about ease of use, screen fidelity, features, affordability, speed and what have you. So just say I’m interested in lots of features, so I go ahead and click on features, it automatically refines and sorts products by the features. The first particular device is the Garmin device here, it has a pretty nice rating, 91 percent of the people are commenting that it’s got positive features. In fact, 92 comments from all over the Web were crawled. I can go take a look at that, so you’ll see we have stuff from eBay, e have things from CNET, Yahoo, and many other sources in terms of comments.

I can actually read the comments, I can even go ahead and look at expert reviews. I can then go ahead and look at product details, once I’m comfortable about the particular product, in terms of the specs.

So this is a nice combination of information, which a search engine can really uniquely do versus just another comparison shopping site, because we’re able to use the entire Web corpus, as well as all of the structured information, and relate it in the context of the buying, or the research task with products. So at this point I’m pretty much ready to buy, and then I can go ahead and transact.

What I want to show you is basically the same experience on the mobile phone as well. I’ll wait for the camera here. So we took the entire product search experience and are making it available as part of Live Search Mobile for anyone who goes to our site,, can get at it from the mobile browser. And it’s the exact same experience. I typed in the GPS query, and I get back a bunch of results. It’s the same products. The one thing that I know that John Battelle blogged about is his whole foods and wine experience, and how he wants to make sure that whenever he buys wine and whole foods, he’s not getting ripped off, and you should look at the prices. You clearly can do something like that here. So here if I’m in a store, I’m looking at this particular device, and if I wanted to even look at the reviews, which probably is a very plausible scenario, because I want to be able to look at the reviews on a particular thing that I’m about to buy in a store. The other thing is, I can look at the price ranges and see if I can get something even cheaper online, so I can go ahead and click on the price ranges here, and you basically have a comparison shopping experience right on the mobile phone. So it shows you the different online sites, and their various prices.

So that’s about extending the same task-oriented experience that we deliver on the PC to the mobile phone, and doing some unique things even on the mobile form factor to sort of cater to the scenarios that are front and center to the end users.

So that was all about really delivering the best results as well as making sure that the results are in the context of a task, and giving the user the power to be able to sort of quickly navigate through the search corpus, and accomplish their tasks. We showed you the product side, we talked about it in this context of travel as well, and those are two areas. And in addition to this, we’re doing things in local, which is another huge domain for us, as well as health is another domain where we are focusing on these deep vertical experiences.

So the thing that I want to then focus on is cashback. Live Search cashback is the new feature that went live late last night and early this morning. The thing that I wanted to sort of describe is the workings of Live Search cashback, and as well as show you the end user experience. The best way to think about paid search, or the search business is as a marketplace, a marketplace that connects end users with advertisers or merchants, with the search provider being the middleman. It’s a $20 billion marketplace today, and we believe that there is a great opportunity as a search provider to innovate in the economic model or the business model, and the way that we are going to go do that is to deliver what we call ad-funded rebates. Take a portion of the money that we collect from advertisers, either CPA or CPC, and redirect that to deliver additional value to end users as cash back, and to deliver additional ROI capabilities to advertisers in terms of paid engagement. That, we believe, will introduce some new dynamics in the marketplace. You can think of this as price competition in an auction marketplace, where we, as a search provider, have the unique ability to be able to change the flows of money, to be able to benefit both end users as well as advertisers.

So what I want to do is show you this in action. So I will sort of take you through the end user experience of Live Search cashback. Similar to the other demos I showed you, I type in digital cameras because I’m in the market for buying a digital camera. So I go ahead and type that query in. I get back an instant answer, which is the cameras, these are the most popular cameras, but what I’m really interested in is actually buying one, and I see an ad here which is being gleaned by Live Search cashback. So this is an example of an ad-funded rebate.

So we took a portion of the advertising money for this particular product from some merchants, and are redirection that as rebates to end users, or cash back to end users. So I will go ahead and click on that glean, just like I would click on any ad, and I’m taken to Live Search cashback. And at this point I can go ahead and select this particular Samsung, which looks like a great camera, to go buy. So I click on that, and it shows me all the retailers that are in the system today who are bidding, in this case on CPA basis, and we are directing this cash back based on the bids. So in this case, this particular camera from Circuit City has 13 percent cash back. That seems like a pretty good price. If I select that, it shows me that the $89 or $90 store price I can get at a bottom line price of $78.29. I can go ahead to the store at this point, and it’s taken me to the checkout process at Circuit City.

We have instrumented our cashback to introduce pretty much no friction into the checkout process. So this is one of the ways we were able to onboard a lot of merchants, and this is something that we’ll learn a ton from, as well, and improve. But overall, you’re now in the Circuit City checkout process, no change to the regular Circuit City checkout process. I can go ahead, add this to the cart, buy it, and once I’ve done that, I can come back and I’ll get a piece of e-mail. With my Live ID, I can log in. So I get a piece of e-mail that states that there is a cashback that I can redeem. I can go ahead and log in using Live ID into my cashback account. And it shows me that I have $400 worth of cashback I’ve earned that I can redeem. It shows all of the cashback I’ve earned from all the products I’ve bought. Ultra Port Series stainless steel tank, I don’t know what that is, but I did buy that. The Sharp Electronics, a nice women’s watch – someone is happy. So in any case, I’ve bought lots of products. The one thing that we do is, we redeem this money within a 60-day period, that’s because of the return policies for merchants, and so you can buy something and within 60-days collect it, after the merchant has released those funds.

So that’s Live Search cashback. So $400 in a day where you have $4 a gallon, who wouldn’t want some cash back? Maybe I can even convince Bill to open an account here. So that’s our cashback experience. We’re very excited about it. As I said, we just launched it last night. We’re getting the first set of users through the system. We’re going to actively look at the experience and the traction we gain. We think it’s got significant value to advertisers. So all of you in the room, we’re looking forward to working with you and engaging with you on evolving this program, and really changing the dynamics of the search marketplace.

So the question really is, how are we going to measure progress here. We’re going to look at three things as we look to the future. We clearly understand that this is a journey. Whenever you change either the user experience or the business model, it takes time to percolate through the behavior changes that we expect from end users. So the first thing we’re going to look at is the number of product offers we have. So what’s the depth and breadth of offers within the cashback metaphor within Live Search. So that’s one. We feel good about the number of products we today have, and Bill will later talk about the total support from merchants to this program. The next one is the advertiser ROI, because that’s the dimension that we care a lot about delivering additional ROI, bringing in the next level of efficiency to the search marketplace. So that’s the next measure in terms of advertiser satisfaction and ROI. And lastly, of course, as this becomes available it will shift share, especially in the commercial domains, and we’ll be tracking that closely.

So those are the three metrics that we’ll be tracking, we’ll be learning from, and we expect to be in dialogue with most of you in the room on how you can take advantage of Live Search cashback to drive the next level of ROI for your online search advertising spend.

That’s all I had. So what I’ll do is, I’ll turn it back to Bill to close up. (Applause.)

BILL GATES: Part of this is the move to really be more measurable, to understand who that customer is, what they’re buying, to reduce the amount of fraud, to go and analyze what types of offers work well. You know, in the digital world, there’s this idea that you target different discounts to different people – here’s a very concrete vehicle that would let you absolutely do that. And then, of course, the end goal is the return on investment. There’s a lot of software analytics to back this system up, and making that integrated in your whole decision process is a key thing for us.

As we say, this is just the start of this, but as we’ve gone around, before the announcement, and talked to people about this, we’ve gotten great support. We’ve actually got 700 merchants that are signed up to participate, and it’s across really all the important e-commerce categories, electronics, books, movies, games, toys, fashion, and as you can see it’s over 10 million different product offers. So overwhelmingly positive feedback from all the partners really confirms that there is this opportunity for change, and that working with somebody who is trying out different things, providing a choice, there’s a lot of interest in that.

One of the lead partners we have on this is eBay. They’ve been great to work with. A lot of things, not only about their product offers, but how we can connect in to PayPal, and how that works. So let’s go ahead and look at a video for eBay to talk about this partnership.

(Video segment.)

So eBay is a key partner, but we have other lead partners who are also doing specific things to integrate in with their system. So let’s go ahead and hear from some additional partners about why they’re participating in cashback.

(Video segment.)

I’m thrilled that these partners are here with us today, and so they’re making a commitment and helping shape this thing, and I want to thank them for their involvement.


Super. I hope they got the photo, if not we’ll stage it again later. So looking ahead, in some ways we’re just at the very beginning of not only the digital revolution, but the specific part about how people engage in buying, or learning, or getting reference information, the way they do that. There is so much more that can be done. And software is a big piece of that, software that often takes many years, even billions of dollars to develop, is central to those experiences. It takes putting together great teams, it takes putting together great partnerships.

So I think what you’ve seen today is this focus on the commercial domain as part of the strategy. This idea of looking at the business model and doing new things, it’s a good illustration of what we’ll be doing. Make no mistake, we’re about having the best search, the best results, so people want to use it for the quality alone, but also some of these innovations in the business model will help excite that and drive that, and deliver the kind of scale that’s important in this business.

So many things to come, a lot of them I the works today, and I’ll be very excited as we roll those out, as well. But, we appreciate you’re being here for this milestone, and being part of the ongoing dialogue with us.

Thank you. (Applause.)

BRIAN MCANDREWS: Bill and Satya, a very exciting new announcement for us here at Microsoft that we’re very excited about. I spoke yesterday about the emphasis on innovation, and it’s exciting to be breaking new ground in the search space in a way that we feel is going to be so beneficial to consumers. As I wrap up here I want to just take a moment or two to try and tie things up from the last couple of days. Before I do that, I’d like to give a quick, but very important, thank you to the folks who worked really hard on putting Advance ’08 together, including ( Steve Cyrix?), Steven Kim, Bob Bejan, Tom Phillips, and all of the folks on their teams, as well as the entire Microsoft Events team. So thank you. (Applause.) No glitches, and we’ve actually stayed on time, which is awesome.

We made a decision, as you know, to build this summit around three core issues, if you will: the consumer connection, strategies for success, and the future media. And I’d like to also thank all the speakers who clearly put in a lot of effort to address each of these pillars with their own unique perspective. I feel like we saw a wide range of views in these areas, and I hope you feel you have good food for thought on these, as well as some action items to take with you.

I also hope you got a feeling for some of the changes that are going on here at Microsoft, symbolized by the launch of our new Microsoft Advertising brand. And if you have three takeaways about Microsoft Advertising, in addition to our pillars of performance, insight, and expertise, I hope they’re these. First, we’re investing really heavily in the digital advertising space, and areas of great opportunity today, like search and display, as well as areas of great opportunity that are emerging, such as video, mobile and gaming.

Second, we’re innovating in this space to help you service your clients better, and whether it’s engagement mapping, which is we believe a very significant advance in the cause of better online measurement, or it’s Live Search cashback, we believe a significant consumer-friendly innovation in the search landscape, we’re focusing significant resources on helping take our mutual challenges, and our mutual opportunities, take on our mutual challenges, and also our mutual opportunities, that we find in the digital world.

Third, and most importantly, we want to be your most valuable partner. We intend to continue to set our priorities based on your input and feedback. So we’re committed to being a leader in the industry when it comes to innovative ways to put our people, our products, and our portfolio to work in partnership with all of you, to create great consumer connections, to develop strategies for success, and to define the future of media.

So in closing I want to thank all of you for taking the time to be with us. We know how incredibly busy everybody is, and so it’s great of you to come, and really important for us to have this kind of time with you, both through these days, and the evenings.

Again, one last plug for the evaluation, please do fill that out. We want to make sure we’re hearing you and that we continue to improve this event, make it effective and useful for all of you. Again, thank you very much, and safe travels to everybody.

Thank you.

Related Posts