Mixed Reality (MR) Experience Design for a Multiverse Travel Game
Mixed Reality | Spatial Design | UX/UI
ImmerX is a mixed-reality project that reimagines storytelling by transforming physical spaces into interactive stages. Instead of passively watching, guests step into dynamic worlds where they actively shape the narrative through exploration and choice. As the UX/UI designer on the team, I led the research, interaction exploration, interface design and the playtesting sessions for the experience.
Mixed Reality
Spatial Design
UX/UI
ROLES
XR Designer
UX/UI Designer
SKILLS
Interaction Design
UX/UI Design
Playtesting
TEAM
1 XR Designer (Me)
1 Creative Producer
1 Developer
3 3D & Tech Artists
1 Mentor
DURATION
12 Weeks
Research Project
@ CMU ETC
PROMPT
Blending physical and digital narrative worlds
Summary
Empowered guests with freedom, interactivity, and engagement to shape their own stories
Leveraged multi-room spaces to enable spatial and environmental storytelling
Seamlessly blended digital and physical worlds to create immersive experiences
As a faculty pitched project, this project served as the starting point for the future development of XR experiences. The prompt challenged us to explore storytelling by turning real-world spaces into interactive stages using Meta Quest 3’s mixed reality, where audiences don’t just watch but actively shape the narrative.
DISCOVER
Balancing creativity with technical feasibility
Brainstorm
Over three weeks, we explored MR storytelling through interaction design, themes, art styles, and AI character chats. Faculty praised our creativity but reminded us to manage scope and address technical limits like rendering delays in procedural environments.
Research
Key Insights
Simple mechanics delight. Grabbing, scaling, or peeking through windows can feel magical when tied to physical space.
Blending realities creates “wow” moments. Seamlessly seeing creatures outside real windows or pulling objects from the floor makes the virtual feel alive.
Space flexibility is critical. Experiences must adapt to unknown room sizes, layouts, and obstacles.
We studied MR games (e.g., Starship Home, First Encounters) and past ETC AR/VR projects. Our focus was on how they balanced interaction, immersion, and space adaptation.
Challenges
High-poly meshes caused delays
Virtual floor blocked passthrough
We faced both technical and design challenges
Multi-room scanning. Quest 3 cannot reliably store multiple spaces, making spatial continuity difficult.
Performance limits. High-poly meshes caused rendering delays; procedural generation risked latency.
User onboarding. MR experiences proved less intuitive than expected, players often missed cues without explicit guidance.
I collaborated with engineers to scale the experience from two rooms to one for stability, worked with 3D artists to use optimized models with rich textures, and proposed a guided onboarding flow to ensure smooth player engagement.
EXPERIENCE
Spatial storytelling that invites playful discovery
Design Goals
Space
Create a whole virtual world outside the actual room
Wow Moments
Value the creation of
magical moments
Immersive
Blend virtual and reality
Intuitive
Direct and indirect controls
Experience Journey
Recommended room set up, top down map
Gesture guide
One Guide
A short onboarding flow sets up the room, registers windows, and teaches core gestures to ground the player.
Travel Console
Vision Gate
Collector
Three Tools
Travel Console for navigating to new worlds
Vision Gate for revealing hidden realities through physical walls and floors
Collector for extracting items from different worlds
Five Worlds
Distinct themed environments with unique interactions, and characters, each designed around memorable “magic moments".
I mapped the end-to-end journey with my teammates, balancing freedom with clarity. We ensured each stage included intuitive guidance and feedback, so players felt oriented while still immersed in discovery.
INTERACTIONS
Immersive worlds anchored by intuitive interactions
Basic Gestures
Basic Gestures
Started with Meta SDK defaults (grab and poke)
Travel Console
A gesture-based hub for navigating between worlds, using enlargement as a primary action with visual and audio feedback.
Collector
Lets players extract items using color-coded UI plus shape and animation affordances.
Vision Gate
Reveals hidden worlds when enlarged and attached to physical boundaries.
Five Worlds Interactions
Playful interactions adaptable to diverse narratives.
We tried about 25 types of interactions before choosing the ones that felt the most fun and realistic to build. I helped brainstorm ideas with the team and documented everything in an interaction catalog.
SPATIAL UI
Interface seamlessly integrated within 3D space
Direct Control
VFX - Indirect Control
The UI used a mix of direct controls (tutorial screens, simple icons, progress indicators) and visual hints (color changes, VFXs, and animations) to guide players.
I designed UI systems that fit naturally into the environment, making the experience clear without breaking immersion. I refined layouts, icons, and visual feedback to help users easily navigate through MR interactions.
PLAYTESTING
Testing, learning and refining
We playtested with students, alumni, faculty, XR developers, and participants with no XR experience in a variety of spaces, including apartments and offices.
The game made me feel …
What single adjective best captures your experience while playing the game?
After playtesting, we found that most players felt happy, surprised, and free, which matched our design goals. However, about 50% of players also felt somewhat lost or confused, especially about how to use the three tools, how to interact with specific objects in each world, and when to turn around after entering a new environment. In follow-up interviews, 30% of players mentioned they wanted more guidance during gameplay.
Iterations
In the earlier version, players often missed the Vision Gate. We updated it to appear only after the handle is triggered, followed by speedline effects during travel. Upon arrival, it now displays “Attach me on the floor” and appears directly in front of the Travel Console for better visibility.
During testing, we found players got tired of leaning down repeatedly to use the collector. We raised its height and added an auto-reset after each use, making it more comfortable and accessible. We also introduced an “All Items Collected” card to clearly signal when players had completed a world and could move on.
REFLECTION
As a designer passionate about XR, this project allowed me to dive deeper into immersive storytelling—exploring its possibilities, current technical limits, and design potential.
Key learnings:
Take initiative as a designer while staying open to teammates’ ideas.
Design for a wide range of players, especially those new to XR; clear guidance and natural UI–environment integration are essential.
Playtesting and iterative design greatly improve the final experience.
© 2025 Yawen Xiao All Rights Reserved
Made with Loves