Mixed Reality Interaction
Also Posted On

How might we improve input and interaction for Mixed Reality?

  • Goals: Pathfind interaction for Mixed Reality
  • Team: Meta Reality Labs: Input and Interaction Team, ISDK Team, and others
  • Role: Product Design/Prototyping

Our team led pathfinding and exploration for new interaction modalities in Mixed Reality, where we launched the first Direct Touch interface.

There was a corpus of past research to stand on. Credit to the ISDK team and others for the First Hand Experience shown below.

This work was developed over a longer time horizon as research, leading to its integration as a primary interaction modality in the operating system:

Our team helped develop prototypes leading to the introduction of microgestures allowing low effort interaction.

Work in this space is ongoing. I designed and cowrote a book collating our learnings:

MDN