In the first week of October, the eyes of the European computer vision community were on Zurich where Microsoft launched the Mixed Reality & AI Zurich Lab. The Lab is led by Professor Marc Pollefeys who has a joint appointment at ETH Zurich and Microsoft.
As Partner Director of Science at Microsoft and Professor of Computer Science at ETH Zurich (Swiss Federal Institute of Technology in Zurich), Pollefeys and his teams benefit from research collaborations between the two organizations in his research domain, Mixed Reality (MR) and Artificial Intelligence (AI). According to Pollefeys, this arrangement is “a real win-win between what ETH and what Microsoft can offer”. At ETH, which is ranked second in the Times Higher Education World University Rankings in Computer Science 2019, Pollefeys works with top students for his research.
At Microsoft he and his team are part of Alex Kipman’s Cognition organization that leads Microsoft’s effort in MR and AI including Kinect and HoloLens products such as the recently announced HoloLens 2. “In this way we can provide great opportunities for students to work with Microsoft, to get access to devices and resources that extend beyond what they have at ETH. There are a lot of open research problems for computer science students to work on with a clear and exciting career path”, says Pollefeys.
Before starting his joint appointment at Microsoft and ETH in summer 2018, Pollefeys had spent two years at Microsoft headquarters in Redmond (USA), working full-time directly under Alex Kipman on HoloLens technology while on leave from his ETH faculty position that he had held since 2007. According to Pollefeys “understanding all sides, academia, industry, as well as the local Zurich context, well in my research field has given me a vision of what I want to achieve with my work, namely building the future of mixed reality using computer vision to map and understand the environment, recognize and track relevant objects, and assist users performing tasks. We are also exploring the synergies between mixed reality and robotics.”
Investment in and for Switzerland
The Mixed Reality & AI Zurich Lab was announced during Microsoft CEO Satya Nadella’s visit to Zurich in October 2018. Nadella had commented on the potential of the new Microsoft research lab in Zurich, especially access to world-class talent whose research could contribute to Microsoft product development as well as advance the science in the field. But he was also curious about any collaboration between the Lab and local industry for exploration of HoloLens and other MR technology in their application scenarios. Microsoft technology should make the whole spectrum of SMEs (Small and Medium-sized Enterprises) to multinationals more competitive and the public sector more efficient suggested Nadella at the time.
The Lab that enjoys close ties to the local Microsoft subsidiary is in an ideal position to make strong contributions especially for Microsoft Switzerland customers. Marianne Janik, Country GM of Microsoft Switzerland commented: “I am very pleased that we can further expand our presence in Switzerland with Marc and his team and that we can bring the close cooperation with ETH, which has existed for over 10 years, to the next level. With the Mixed Reality & AI Zurich Lab, we can not only support basic research at ETH Zurich, but also support Swiss organizations in the development of innovative products and services.”
Close collaboration with ETH Zurich
Over the past year, the Lab has almost grown to its initial target size of 15 full-time employees and is located in a new space in the heart of Zurich city center, less than 15min from Pollefeys’ ETH Lab – the proximity allowing frequent interaction and ongoing collaboration. Pollefeys splits his time between his ETH and Microsoft offices and supervises research teams at both. Besides close collaboration between the respective teams, the Microsoft and ETH researchers participate actively in the wider academic ecosystem and use the Swiss Joint Research Center and Inria Joint Center to drive open and collaborative research projects with key academic collaborators at ETH, EPFL, and Inria in France.
Various Lab opening activities
The Lab’s launch event took place from 2nd to 4th October in Zurich. For the opening, many Microsoft experts traveled to Switzerland, including special guest Alex Kipman, the inventor of HoloLens and Kinect, as well as Microsoft Research (MSR) Cambridge Lab Director, Christopher Bishop, and MSR Redmond Lab Director, Donald Kossmann. Several prominent computer vision researchers from leading European research institutions and from Microsoft Research and Cognition also honored us with their participation.
One highlight of the launch was Alex Kipman’s public talk at ETH. Alex gave an ETH Global Lecture entitled “HoloLens 2 – Unpacked” where he described the engineering and research accomplishments that went into the HoloLens 2 to approximately 400 ETH students and faculty in ETH’s main lecture theatre, the Audimax. The audience’s excitement was palpable, and many attendants stayed on to talk to Alex and the Cognition team during a subsequent drinks’ reception. The recording is accessible on the ETH event page.
The main component of the launch was a scientific workshop with talks from Microsoft researchers and ETH Zurich and other academic collaborators, especially from our the Swiss Joint Research Center and Inria Joint Center. There were considerably more attendants than expected. Of the 110 attendants, about a quarter were from Microsoft, half from ETH Zurich, and others joined from EPFL, HWTK Leipzig, Inria, MPI, TU Munich, and University of Zurich. The agenda was a mix of presentations from Microsoft and academic collaborators. For more details, please check the workshop event page.
In addition, two evening receptions were held in the new Lab, the first was organized jointly with Microsoft Switzerland and the second was part of the workshop. On 2nd October, the Microsoft Switzerland team had organized a VIP Reception with invitees from Swiss industry, politics, and academia. Among the guests were senior representatives from companies such as ABB, Balgrist University Hospital, ESRI, Hilti, Omya, Schindler, Stadler Rail, and Zuhlke as well as from ETH Zurich. There were demos to the visitors. Microsoft Switzerland Country Lead Marianne Janik and Pollefeys gave short speeches.
The Lab’s research journey has journey has only just begun. While research can often lead in unexpected directions, the three research themes at the outset are:
- Crowd mapping to allow sharing and persistence of Holograms at world scale. This work directly links to Azure Spatial Anchors, but also looks at longer term challenges such as privacy preserving image-based localization methods.
- Spatial intelligence for autonomous robots and human-robot interaction.
- First-person activity recognition.
At the end of an exciting week, Pollefeys acclaimed “It has been a great start and I could not be more excited about the research journey upon which we have embarked in Zurich. We have an incredible group of people full of ideas and passion, and I am extraordinarily optimistic about what we can achieve here.”
Interested in more information? Please visit the following links:
- Website of Mixed Reality & AI Zurich Lab
- Lab opening event
- ETH Zurich (i.e. ETH announcement)
- Global Lecture of Alex Kipman at ETH Zurich (i.e. Global Lecture)
- Swiss Joint Research Center
- Inria Joint Center
- Microsoft HoloLens
The title picture of Alex Kipman was also made by Andreas Eggenberger on behalf of the ETH.
