I am an XR designer with architecture background, working from SF.
Spectacular fireworks through your roof.
A magical portal that opens to the boundless night sky. A mixed reality experience that interacts with our familiar world.
Mixed Reality headsets unlock numerous use cases that connect the virtual spaces with our surrounding. One of them is to embed a portal to another world. This mixed reality experience enables the users to interact with the physical spaces in their living rooms and set off fireworks into the unbounded sky beyond the ceiling.
The project focuses on the new opportunities / questions in Mixed Reality, compared to VR experiences:
I chose to set the stage in the living room, one of the most common spaces to use a headset. Due the nature of mixed reality, the environment can only be loosely defined rather than clearly configurated. So, I anchored the MR experience on a couple of indispensable architectural elements in a living room — a ceiling and a table.
With the spatial constraints in mind, I use sketches to quickly ideate and record key interactions and user flow.
The first step is to create a portal of a night sky in the ceiling. Raycast is the best interacting method due to the distance. The key is to convincingly blend the portal with the ceiling. Details in the prototype section below.
The second step is to spawn the fireworks. After multiple experiments, I settled down with a simple tap of hand on the table. The physical interaction that creates virtual objects is a powerful affordance to blend the virtual reality and the physical space.
The firework will launch into the sky after a few seconds, just like a real firework rocket would.
To create a spectacular show of firework, a boundless night sky is essential. However, the area of a living room's ceiling is rather limited. So, I need to create a portal that fits in a typical ceiling and houses a night sky with large depth. I achieved this by using stencil shaders in Unity to mask any gameobjects outside the portal.
The mixed reality experience was built on top of Unity AR Foundation with Meta OpenXR support. The configurated physical space information is passed to AR foundation by OpenXR as AR planes.
AR Foundation in Unity handles the plane detection of our interactables, the ceiling and the table.
Raycast on the ceiling to spawn the night sky portal.
Direct interaction by hand to spawn the firework then launch it into the sky.