Q&A: Microsoft’s Anti-Spam Technology Roadmap

REDMOND, Wash., Feb. 24, 2004 — In his keynote address at RSA Conference 2004 in San Francisco, Microsoft Chairman and Chief Software Architect Bill Gates today outlined Microsoft’s technological vision for addressing the spam problem, including the company’s Coordinated Spam Reduction Initiative (CSRI) and a technical proposal for e-mail that’s being likened to the phone industry’s caller-ID system.



Ryan Hamlin, General Manager, Anti-Spam Technology and Strategy Group.

To learn more about Microsoft’s recommendations and understand how they fit into the company’s broader anti-spam efforts, PressPass spoke with Ryan Hamlin , general manager of Microsoft’s Anti-Spam Technology and Strategy Group.

PressPass: What is Microsoft announcing today?

Hamlin: In keeping with the investments we’ve already made to address the spam problem and our continued engagement with others in the Internet e-mail community, we’re outlining a technology strategy to help end the strain that spammers put on e-mail networks and users’ Inboxes. This strategy includes technical recommendations for industry consideration and a commitment from Microsoft to move forward swiftly in helping to find real solutions.

PressPass: Why is Microsoft doing this?

Hamlin: We view e-mail as an essential communications tool, and we want to make sure it stays that way. We hear from our e-mail customers that spam is their No. 1 concern, and as a technology leader, we feel a responsibility to drive innovation and invest as needed to help resolve that concern for them. Plus, spam is a severe and growing problem that strikes at the heart of our business. Microsoft is well positioned to take a leadership role on the spam issue — as an ISP and e-mail provider, as a builder of enterprise-level e-mail clients and as a commercial marketer of e-mail software. So it makes sense for us to devote technology resources toward a solution. We believe that it will require the coordinated efforts of the entire e-mail community to make significant progress in deterring spam so that computer users everywhere can experience much less unwanted e-mail correspondence.

PressPass: How does today’s announcement fit into Microsoft’s overall efforts to combat spam?

Hamlin: We’ve consistently advocated a multi-pronged, holistic approach to the spam problem that includes ongoing technological innovation, industry cooperation, consumer education, effective legislation, and targeted enforcement. Microsoft is committed to using its resources to help address the spam epidemic from each of these perspectives until we see evidence that the problem has substantially subsided.

PressPass: What specific technology recommendations is Microsoft making?

Hamlin: Besides continuing to invest in innovative approaches, such as the machine-learning technology available in our SmartScreen spam filter and emerging proof-based approaches that Bill Gates mentioned recently, we’re releasing a technical proposal for industry consideration called the Coordinated Spam Reduction Initiative (CSRI). CSRI includes a roadmap for policy and technology infrastructure changes that can help stop the scourge of spam. One of the first technical recommendations we outline in CSRI is a “caller ID” type of approach that takes aim at the rampant practice of sending e-mail with forged “From” addresses, commonly called spoofing.

PressPass: Can you describe the “Caller ID” approach in simple terms and explain how it works?

Hamlin: Caller ID is Microsoft’s recommended next technological step to deter spam. Essentially, it’s a mechanism for legitimate senders of mail to help ensure their Domain Name is not being abused by a spammer. In a nutshell, Caller ID involves two key steps. One, senders of e-mail publish the IP [Internet Protocol] addresses of their outgoing mail servers in DNS [Domain Name System] in an e-mail policy document. Two, the e-mail software at the receiving end of a message queries DNS for the e-mail policy and determines the “purported responsible domain” of the message. This is done by comparing the information in DNS to ensure it matches the information on the originating mail. We believe this technical solution gets at the root of the spam problem by helping to confirm legitimate senders.

PressPass: What’s the status of the Caller ID technology?

Hamlin: The proposed specification is currently in draft-for-comment form, posted at www.microsoft.com/spam, and we are open for business in getting feedback on it from all key e-mail stakeholders. We invite and encourage industry-wide input and scrutiny.

PressPass: How long has Microsoft been working on Caller ID technology?

Hamlin: The Caller ID concept has been in development at Microsoft for about a year and is one of several proposals that have been considered within the industry for some time. Today we are announcing that we are rolling out a pilot program of our Caller ID for Email proposal for evaluation purposes and we welcome others to join us in this test. We want to get as broad of feedback on our proposals as possible, as part of our ongoing effort to work with other key email stakeholders to discuss new ideas and approaches to help put an end to spam, so we are publishing our technical specs for CSRI and CallerID for public feedback at www.microsoft.com/spam.

PressPass: How do you see Caller ID for e-mail being implemented?

Hamlin: The infrastructure for Caller ID is designed to be implemented at the service provider level. At Microsoft, for example, we’ll be deploying it first in Hotmail, and we’re also working with others such as Brightmail to test the spec as well. We will begin publishing outbound IP addresses immediately for Hotmail. Inbound processing will take a little more time to implement, but will begin early this summer.

PressPass: Does Microsoft believe that this Caller ID solution will stop the spam problem?

Hamlin: In and of itself, no. We recognize that no single technology will make the spam problem go away entirely. And while we hope Caller ID will serve as a deterrent, it doesn’t actually prevent spam from being sent. Remember, one of the key benefits of Caller ID is to help legitimate senders of mail protect their investment and brand in their Domain Name by making it very difficult for spammers to try and spoof their domain. Additionally, we believe it provides a sound basis for other measures that will be able to do an even better job at detection. Also, remember that Caller ID is part of a larger picture as one of the underpinnings of CSRI.

PressPass: Can you elaborate a bit on the Coordinated Spam Reduction Initiative?

Hamlin: The CSRI specification is Microsoft’s comprehensive technology and policy proposal for reducing spam. It’s intended as a concrete plan of action — an outline for the industry, if you will, of some simple enhancements to e-mail distribution that the Internet community can adopt on an incremental basis. The approach we advocate in CSRI consists of several parts. First is establishing verifiable identity in email through a caller-ID approach, such as the mechanism we outlined today. Second, it’s important to enable high volume email senders to demonstrate their compliance with reasonable email policies. Third, we need to create viable identification alternatives for smaller-scale email senders.

PressPass: What’s the status of CSRI?

Hamlin: Like the Caller ID spec, the CSRI spec is in draft-for-comment form and posted online at www.microsoft.com/spam. We’re actively soliciting comments, questions, suggestions and other feedback on it in the belief that CSRI will only be strengthened though such a process — an end that will benefit the entire Internet e-mail community.

PressPass: What anti-spam technologies does Microsoft support as a part of its technology vision?

Hamlin: Besides Caller ID and our SmartScreen Technology, we also believe safelists and emerging proof-based approaches such as challenge/response, computational puzzles and micropayments can provide a strong challenge to spam.

PressPass: Can you explain SmartScreen Technology and tell us how it works?

Hamlin: SmartScreen Technology is an intelligent spam-filtering solution based on Microsoft Research’s patented machine-learning technology. In essence, SmartScreen Technology learns how to distinguish between legitimate e-mail and spam by synthesizing extensive user input and by applying probability-based algorithms to incoming messages. Hundreds of thousands of volunteer Hotmail subscribers are “training” the SmartScreen Technology to recognize markers in both types of e-mail to increase overall accuracy and lower the likelihood of false positives. With SmartScreen Technology working in Hotmail, we know that upwards of 90 percent of spam is blocked so it never reaches the subscriber. Once SmartScreen Technology is fully implemented, our customers should encounter noticeably less spam on all the Microsoft e-mail platforms they use, including MSN, Outlook, and Exchange Server.

PressPass: What are safelists and how do they work?

Hamlin: Safelists are a regularly updated collection of verified legitimate senders that ISPs and e-mail recipient filters can check against to determine whether a message should be delivered. At the recipient end, for example, safelists allow an e-mail user to specify certain addresses or domains that he or she considers safe. Then, if an incoming message contains an address or domain on that person’s Safe Sender or Safe Recipient list, the message would not be subjected to filter scans.

PressPass: What’s challenge/response and how would it work?

Hamlin: Challenge/response is an innovative proof-based method of validating the legitimacy of an e-mail message to determine that it’s not spam. In a challenge/response scenario, the e-mail recipient sends some form of challenge, and the e-mail sender has to respond in the correct manner. This could be done through a number of approaches, such as Human Interactive Proofs, or HIPs, which can test whether the entity on the other end is a person or a machine. Like other proof-based approaches, challenge/response is grounded in the idea that the solution to spam lies in changing the economic incentives for sending it.

PressPass: How would computational puzzles work to curb spam?

Hamlin: Computational puzzles require a sender — or a sender’s computer — to help validate the legitimacy of an e-mail message by solving a simple puzzle, using CPU resources. This activity would take little-to-no time for a regular e-mail sender’s computer to do, but in theory it would overwhelm the computing cycles of someone sending large amounts of bulk e-mail. Again, the goal is to minimize the economic viability of spam.

PressPass: What are micropayments and how might they work?

Hamlin: Essentially, micropayments entail making spammers pay for each e-mail message they send, just like people pay for postage on paper mail. This kind of proof could take the form of a financial payment or a computer payment, such as the computing cycles outlined in the computational puzzle-style approaches. Either way, the idea is to create an effective financial deterrent to sending spam.

PressPass: How do Microsoft’s anti-spam initiatives fit in with broader industry efforts to combat spam?

Hamlin: We’re continuing to work actively with others in the industry, including other technology vendors and ISPs, and other e-mail stakeholders to drive toward technical standards and promote collaboration in the development of industry guidelines that address the spam problem. We understand the importance of working together on a range of solutions to encourage adoption of solid approaches that will help put an end to spam. Overall, we’re encouraged with the progress that’s been made to date, but we also recognize that the industry as a whole has miles to go before spam is eradicated.

PressPass: What’s Microsoft’s position on some of the other anti-spam technical proposals circulating today?

Hamlin: We’re pleased to see companies bring their unique perspective and ingenuity to the table in a coordinated effort to solve the spam problem. Our goal is to be a good partner for all technical specifications and industry developments. And while we’re convinced that our own Caller ID proposal offers the most benefits to the e-mail community, we believe that several proposals under consideration within the industry today have some technical merit.

PressPass: What can we expect to see from the industry in the last year to address the spam problem?

Hamlin: We will see a lot of industry and individual company progress this year on a variety of fronts, including improved filters and ideas on Internet Protocol changes and implementation, such as the caller ID for e-mail approach. We’ll also see more industry and government collaboration — on best practices, on information sharing, and on enforcement cases. There is a whole lot of work that remains, to be sure. But we’re excited about the avenues yet to be tested and confident that it will lead to solving this issue for consumers.

PressPass: When does Microsoft think the spam problem will be solved?

Hamlin: Microsoft is focused on working with others in the industry to help solve the problem swiftly. We believe that innovation and technological progress, combined with a holistic approach to industry cooperation, consumer education, legislation and enforcement will indeed help contain the spam problem within the next 24 months.

PressPass: Where can people find out more about anti-spam measures?

Hamlin: We encourage people to visit http://www.microsoft.com/spam for tips and information on how they can help stop spam. MSN also hosts a number of educational Web sites for local regions around the world. These sites provide information in more than 30 languages to help users worldwide understand how to protect their systems and data.