ETH Zurich - D-INFK - IVC - CVG - Lectures - Mixed Reality

Mixed Reality


MR Logo
Course ID: 263-5905-00L (Autumn Semester 2022)
Instructors: Dr. Iro Armeni, Prof. Marc Pollefeys
Teaching assistants: Taein Kwon, Jonas Hein, Julia Chen, Tianxu An
Lectures: Mondays 10:15-13:00 in CAB G 11
Language:English
Credits:5KP

Quick Access

All course materials are provided in Moodle.

Course Description

Brief Description The goal of this course is an introduction and hands-on experience on latest mixed reality technology at the cross-section of 3D computer graphics and vision, human machine interaction, as well as gaming technology.
Learning Goals After attending this course, students will:
1. Understand the foundations of 3D graphics, Computer Vision, and Human-Machine Interaction
2. Have a clear understanding on how to build mixed reality apps with a lot of hand-on experience
3. Have a good overview of state-of-the-art Mixed Reality
Course Evaluation 1. 10%: Project Proposal presentation and report
2. 20%: Midterm Project presentation and report
3. 70%: Final project which includes a report and presentation/demo
Prerequisites / Notice Prerequisites include:
- Good programming skills (C# / C++ / Java etc.)
- Computer graphics/vision experience: Students should have taken, at a minimum, Visual Computing. Higher level courses are recommended, such as Introduction to Computer Graphics, 3D Vision, Computer Vision.
Related Courses 151-0306-00L "Visualization, Simulation and Interaction - Virtual Reality I"
151-0317-00L "Visualization, Simulation and Interaction - Virtual Reality II"
252-0570-00L "Game Programming Laboratory"

Course Schedule

No. Date Topic Presenter Tutorial/Short Lecture
1. 26.09 Introduction to Mixed Reality and Course Logistics Iro Armeni, Marc Pollefeys (ETHZ, CVG) -
2. 03.10 Mixed Reality with HoloLens 2 Marc Pollefeys (ETHZ, CVG) Tutorial, Patrick Misteli (Microsoft)
3. 10.10 3D scene understanding for AR applications Federico Tombari (Google, TUM) Tutorial, Patrick Misteli (Microsoft)
4. 17.10 Egocentric Vision to Understand and Predict User Behavior through Wearable Devices Antonino Furnari (University of Catania) Tutorial, Patrick Misteli (Microsoft)
5. 24.10 Project Proposal Presentations - -
6. 31.10 The Wanderings of Odysseus in 3D Scenes Yan Zhang (ETHZ, VLG) -
7. 07.11 Mid-Term Project Presentations - -
8. 14.11 Augmented Virtuality, Diminished Reality, Substitutional Reality Andreas Fender (ETHZ, SIPLAB) "Evaluating User Experience in MR", Tiffany Luong (ETHZ, SIPLAB)
9. 21.11 Computer Vision Capabilities on Meta VR Headsets Mariano Jaimez (Meta) -
10. 28.11 Medical Augmented Reality Hooman Esfandiari (Balgrist UniversitatsKlinik) -
11. 06.12 Designing with Mixed Reality Anton Savov (ETHZ) "AR as a Tool for Productivity Improvement in Construction", Timothy Sandy (Incon.ai)
12. 12.12 TBD Arzu Çöltekin (FHNW) -
13. 19.12 Final Project Presentations & Demo - -

Deadlines

  • October 10: Group formation and project selection - Students select from a list of project proposals and we assign them to the topics.
  • October 20: Project proposal report - Students submit their project proposal reports after discussing with their assigned supervisors.
  • October 24: Proposal presentations - Students present their project proposals during lecture.
  • November 7: Midterm presentations - Students present their progress on their projects during lecture.
  • November 14: Midterm project report - Students submit a written report with current updates and plan.
  • December 19: Final project presentations - Students present their projects in a joint session.
  • December 19: Final project demo - Students present their projects in a live demo session.
  • January 06: Final project reports - Students submit their final reports for the projects.
* All the submission deadlines are due by 23:59 on the specified date.

Tutorials

To get you started with developing mixed reality applications, we will host tutorials at the beginning of the semester. These will take place in CAB G 11 after the main lectures. If you want to simultaneously implement the tutorial, you can install Unity for your OS. In case you have trouble with this or do not have access to a laptop, you can use the student lab CAB H 56, which has been reserved for this purpose. We will be livestreaming there the tutorial (the room has 24 seats). The machines in the lab are all equipped with Windows and Unity. When there is no tutorial after the main lecture, students are highly encouraged to continue working on their projects. The TAs will be around in case you have questions to ask.

Student Projects

Over the semester, students will work on a project related to a topic in Mixed Reality in collaboration with a supervisor. Students are required to form groups of 4 and submit their preferred project topics first. For this purpose, we will provide a list of project suggestions, but you are free to propose your own project.

For your projects, we can make the following hardware available:
  • Microsoft HoloLens
  • Oculus Rift
  • Mobile phone devices
  • Webcams

Students with project ideas that require any other special equipment, please talk to us!

Project Proposal: Each student group is then required to hand in and present a project proposal by the announced deadlines. Make sure to talk to your assigned supervisor and discuss the project with them while planning your proposal. Each team will present their project proposal during a designated lecture. The template for the project proposal report can be found here.

The proposal should be 1-2 pages describing what you want to do in the project, and how you plan to achieve your envisioned results. A good idea is to identify the algorithmic and technical challenges within the project. Try to address each of them individually and explain your considered solutions; also make an attempt to think about alternatives if you believe a particular approach is unstable or likely to fail.

Midterm presentations: At this milestone you present what you did so far so that you get feedback. You are encouraged to raise open questions. This is a possibility for us to steer the project and help you, as well as to get feedback from your fellow classmates. In addition, you are required to submit a 2-page report that identifies your current status, problems you have identified, and your next steps.

Final presentations: They will be held as a regular presentation session. In addition, you are required to hand in a technical report for your project. The report format should be in parallel with CVPR paper format. Latex and Word templates can be found here.

Final demos will be held right after the end of the final presentations in a space nearby. More details on this will follow soon.

Discussion Forum

There exists a discussion forum page in MOODLE for this course. Students should sign in using their ETHZ accounts and participate in the discussion forums. Please put all your discussions related to the lectures, paper presentations and projects there.


© CVG, ETH Zürich webmaster: psarlin@inf.ethz.ch