REDMOND, Wash., July 26, 2000 —
“The Perfect Storm”
and a slew of other recent movies and video games have done for computer graphics what the Model T did for cars: they have opened the eyes of the general public to the potential of new technology.
Surface Light Fields for 3D Photography
But now that technology has achieved such lifelike effects, the role of more traditional approaches to visual expression — composition and design, shading and detail — becomes less certain. Will these more subtle forms of art become the horse and buggy of the computer age?
Not if David Salesin can help it.
For more than a decade, Salesin, a senior researcher at Microsoft Research in and associate professor at the University of Washington in nearby Seattle, has helped push the frontiers of the graphics revolution that is making it increasingly hard to distinguish computer-generated images — also called “photorealistic graphics” — from the real thing. But Salesin has distinguished himself in another way as well: by ensuring that old-fashioned aesthetic principles are not abandoned in the rush forward.
Thanks to Salesin and his colleagues, computer users can now more easily create intricate pen-and-ink drawings and watercolor paintings on their desktop computers. They also can create interlocking picture tiles like those of Dutch graphic artist M.C. Escher; easily decorate documents and letters with algorithmically created floral designs; or transform their online chat sessions into cartoons, complete with animated characters and dialogue balloons.
This week, the Special Interest Group on GRAPHics will recognize Salesin for these and other contributions in the area of non-photorealistic graphics when it awards him the ACM SIGGRAPH Computer Graphics Achievement Award. He will receive the award at the SIGGRAPH 2000 computer graphics conference in New Orleans, which runs through July 28.
Along with being the sixth Microsoft researcher to receive a SIGGRAPH award since the program’s inception in 1983, Salesin co-authored five of eight research papers by Microsoft researchers that will be published at this year’s conference.
“Salesin’s work on computer-generated pen-and-ink illustrations and, subsequently, computer-generated watercolors are landmarks in this emerging field,”
said Ed Catmull, chairman of SIGGRAPH’s awards committee and co-founder of Pixar Animation Studios.
“For the past six years, his publications in this area have been extremely significant and influential.”
In addition, Salesin is recognized as a top teacher and mentor, who is helping to forge strong bonds between academia and the business world. His dual existence at the University of Washington and Microsoft, colleagues say, allows him to spur joint research and seminars and cross-pollination of ideas between graduate students and Microsoft researchers.
Much like the floating span of Highway 520 that crosses Lake Washington and connects Redmond to Seattle,
“this relationship creates a tremendous bridge between Microsoft Research and the University of Washington,”
said Ed Lazowska, head of the UW’s Department of Computer Science & Engineering.
As a result of this growing partnership and the wealth of talent at Microsoft and the UW,
“one could argue the national center of expertise in computer graphics has moved to the Puget Sound region,”
Lazowska said.
Bitten By a Flying Hypercube
Salesin, 39, can pinpoint the exact moment when he was bitten by the computer graphics bug. It was 1979, and he was attending an early admissions day at Brown University in Providence, R.I. He sat in on a presentation math professor Thomas Banchoff gave to woo students to his program. It featured a graphic of a computer-generated, four-dimensional cube that gyrated and spun on the screen as if it were tumbling through the air in front of them.
“Escherization” techniques allow computers to understand and generate interlocking tiled images similar to those found in the work of artist M.C. Escher.
“I was just completely hooked,”
Salesin said, raising his quiet voice slightly.
“I’d never seen anything like that. I was completely fascinated.”
Salesin was also pretty unique, Banchoff said. Not only was he a math and computer graphics wiz, but he also loved art and music, even performing in an a cappella singing group whose repertoire included ABBA’s
“Take a Chance on Me.”
“He wasn’t the prototypical computer geek,”
said Andy van Dam, Brown professor of computer science and technology and education and one of Salesin’s undergraduate mentors. Van Dam is currently a member of Microsoft Research’s Technical Advisory Board.
The Rochester, N.Y., native’s talents and interests took him in numerous directions. He majored in computer science but seriously considered other options, even going as far as taking entrance exams for medical school.
Salesin chose computer graphics after van Dam urged him to choose the profession that would hold his interest and allow him to use all of his talents.
“I seduced him to the ‘Dark Side,'”
van Dam joked recently.
He’s not far off. After graduating magna cum laude from Brown in 1983, Salesin launched his career with
“Star Wars”
creator George Lucas’s Lucasfilm studio, where he contributed graphics to the feature-length film
“Young Sherlock Holmes.”
He went on to work for Pixar Animation Studios, where he worked on the Academy Award-winning short film
“Tin Toy,”
as well as Sogitec Audiovisuel in Paris and the Digital Equipment Corporation’s Paris Research Lab, before earning his doctorate from Stanford University. His thesis was on computational geometry. He also spent a year as a visiting assistant professor at Cornell University, researching new algorithms in photorealistic rendering, before taking a position on the UW faculty in 1992.
Making Graphics Less Realistic, More Customizable
But Salesin’s interest in traditional arts and craftsmanship soon began to infuse much of his research.
“I really love old things,”
he explained, sitting in his Redmond office, furnished with an antique office chair and vintage electric fan.
”
There is such a rich tradition to draw upon in the visual arts, so much that has been invented and discovered in areas like graphic design, typography, ornamentation and cinematography.
Environment Matting Extensions: Towards Higher Accuracy and Real-Time Capture
“Until now, a lot of this tradition has been overlooked or ignored when creating graphics by computer, whether for electronic publishing or computer games,”
he added.
“The computer should provide an opportunity to bring this craftsmanship back in, to draw upon these rich traditions to create graphics that are automatically and perfectly tailored for the needs at hand.”
Pen-and-ink drawings may be one of the best examples. Although many publications may consider the art form old fashioned, medical atlases and automotive manuals continue to use hand-drawn illustrations because these drawings allow the artist to focus attention on specific details and omit others that might distract the eye.
But the ability to create pen-and-ink style drawings with computers did not exist before Salesin and his colleagues developed methods to do so. The central challenge: how to create detail, or texture, without creating too much darkness, or tone. They crafted ways to prioritize strokes until an image reached the desired tone and to create a desired texture without drawing every virtual pen stroke. Another method provides a library of different stroke textures, which can be used to convert photos or other images into pen-and-ink illustrations.
To translate the delicate hues and shades of watercolors, Salesin and colleagues had to approximate many of the physical processes that occur when pigment, water and paper interact. After consulting with chemists, fluid dynamics experts and a Seattle art supply company, they simulated the effect of an artist applying multiple washes of color, as well as the appearance of paint flowing over different paper textures and areas of varying dampness. They even recreated the edge-darkening effect that occurs when watercolor paint spreads into a dampened area — an effect that gives watercolor what artists call its
“luminous quality.”
To create computer watercolors, artists can adjust the physical parameters for each glaze with on-screen toggles. They also pick the color of the pigments and size of the virtual brush. Those with less ambition or time can set a few basic parameters and then let the computer automatically
“watercolorize”
an image.
Forging New Ground, Filling Holes
The UW’s Lazowska likes to call Salesin’s work
“deliciously retro.”
But he doesn’t think his colleague’s ability to look back limits his ability to look forward.
“The conservative thing to do is to make progress in the direction everyone else is going,”
Lazowska said.
“If you are talented, you make progress a little faster. David spends all of his time looking for new directions. And suddenly, hundreds of researchers follow his lead. He blazes entirely new paths.”
“He’s clearly the leading figure in computer graphics in his generation,”
Lazowska added.
Michael Cohen, a fellow Microsoft graphics researcher who in 1998 received the same SIGGRAPH award as Salesin, explains his colleague’s talents a little differently:
“He’s the kind of guy who is able to look at the world, see gaps in our knowledge, and fill them.”
Salesin has begun filling some of those gaps in Microsoft products since he began working at the Redmond campus in 1999. They include a method for more easily searching for clip art, illustrations and photos, one of the technologies Microsoft acquired when it purchased Numinous Technologies, a start-up Salesin founded to develop commercial applications for his research. The clip-art finder is featured in Windows 2000.
Comic Chat, the program that turns computer chat into comic strips, is offered in several versions of Internet Explorer, Windows 98, and Windows NT4. It’s also available in 24 languages. Salesin’s work with type fonts allows Microsoft typographers to create new styles of more easily readable lettering — a painstaking process that can take a week per font — in a fraction of the time.
In the near future, Microsoft’s virtual sports games may soon have more realistic crowd scenes thanks to Salesin and fellow researchers’ development of
“video textures,”
a type of videotape loop that creates mini-movies that don’t appear to repeat. He is also helping develop ways to create more realistic, three-dimensional human faces on computers and to train computers to work as virtual cinematographers for PC movies and video games.
The impact of Salesin’s research is felt far beyond Microsoft. He has become one of the most published researchers in computer graphics. He has co-authored a book on wavelets — a mathematical tool that allows researchers to better analyze images — and dozens of research papers, including 29 at SIGGRAPH, the most prestigious organization to publish graphics research. Peer researchers select roughly a sixth of the papers submitted each year to SIGGRAPH for publication, based on the quality and importance of the work, Catmull explained.
Salesin’s papers on video textures, computer typography and
“Escherization”
will be presented at this year’s conference. These, as well as two other papers he co-authored, will appear among 59 selected from about 400 papers submitted this year.
“It’s not easy to get published at SIGGRAPH,”
Catmull said.
Teaching, Building Bridges
Banchoff, the Brown professor whose hypercube captivated Salesin, thinks his former student’s greatest contribution may not be his research, but rather his ability to capture the imagination of the next generation of graphics researchers.
“He’s not just a great graphics researcher,”
Banchoff said.
“He’s a great teacher.”
Salesin has received awards from The Carnegie Foundation for the Advancement of Teaching and the Council for the Advancement of Teaching. In 1997, the UW awarded him its Distinguished Teaching Award, a prize given to four people a year on a campus with 3,000 instructors.
Salesin created a unique sequence of undergraduate computer animation courses that brought together students from art, music and computer science to help them learn both the principles of computer animation and how to work together before they get hired by computer animation companies. Lazowska knows of no other college or university with a similar program.
Salesin also brings in professional animators to help teach some of his classes and speed up the progress of his students. One video created by UW undergraduates won the Audience Prize in the Rio de Janeiro Animation Festival. The students had only 10 weeks of training in a class devised by Salesin and led by Cassidy Curtis of Pacific Data Images.
Colleagues also note how Salesin mentors many of his students by involving them in his research at the UW and Microsoft. The experience is vital for students. It also allows him to juggle more research projects and brings students and Microsoft researchers together to share ideas.
“I wouldn’t feel as satisfied doing research all by myself,”
he explained.
Salesin said he’s equally pleased with his dual existence at Microsoft and the UW. He works one day in his Redmond office and the next on the Seattle campus, pursuing his research largely unencumbered by deadlines and product demands, he said.
Lazowska said the shared research, seminars and other partnerships fostered by Salesin’s dual existence provide
“incalculable advantages”
to the university. The student input and ideas also provide Microsoft a competitive advantage.
“It’s an ideal situation from our viewpoint,”
he said.
Although he values both of his workplaces, Microsoft provides some things academia doesn’t, Salesin said. He can pursue research that interests him and then get any results with commercial applications into products that people use every day of their lives. He also gets to work with some of the best minds in computer science and other fields, such as the Microsoft typography gurus he is collaborating with on new methods for creating fonts.
“There are so few people in the world at that level of expertise,”
Salesin said.
“This arrangement provides me a wonderful opportunity to collaborate with people I would otherwise never get to do work with.”
Giving Back
Although Salesin still has decades of research ahead of him, he thinks it’s time to begin giving back. He wants to repay the many artists who have assisted him with his research by helping them use computers in their own creative endeavors. He hopes that such collaborations may eventually lead to the creation of new forms of art that might include interactive or interpretive opportunities, motion or Web connections. Ultimately, he wants to help artists create
“things that would be considered masterpieces hundreds of years from now but that could never have existed without the computer.”
So much for being deliciously retro.
