Inside this project
Problem & Overview
“You have broken assumptions and broken software.”A group of blind artists and technologists to Microsoft on the state of VR accessibility
While 2D platforms benefit from decades of accessibility standards (semantic HTML, ARIA, screen readers), immersive 3D environments have almost none. Engines like Unity and Unreal don't expose content to assistive technologies, and no established patterns existed for navigating or understanding 3D scenes nonvisually.
I led a multi-phase initiative at Microsoft to define new paradigms for spatial accessibility from the ground up, using Microsoft Mesh as the testing and shipping surface. Mesh was Microsoft's premier platform for immersive collaboration at the time.
An introduction to immersive collaboration in Microsoft Mesh.
Research
I started by deeply studying how blind users navigate 2D systems — using screen readers daily, reviewing accessibility research, and collaborating with internal ERGs and experts across Xbox, Windows, and Mixed Reality.
Hands-on experience to understand user workflows and pain points
Analysis of existing accessibility studies and best practices
Partnership with ERGs and experts across Xbox, Windows, and Mixed Reality
Cross-Functional Collaboration
I led design and technical prototyping in Unity, shaping the initial feature set. This work relied on a cross-functional team including:
- Accessibility PM and technical designers
- Audio engineers and shader specialists
- Engineers who built performant, shippable systems
- User researchers running iterative studies
- Internal subject-matter experts, including blind engineers and designers
It was a highly collaborative and advocacy-heavy effort, especially since accessibility wasn't an org priority at the time.
Reimagining Accessibility for Immersive Environments
I continued to lead the work here as my amazing partners and I prototyped and shipped several innovative features in an iterative development cycle with consistent feedback loops.
High-Contrast Filters
Two complementary systems for low-vision users
- Environment Filters: shaders highlighting floors, walls, and fixed geometry to aid spatial awareness
- Interactive Outlines: distinct visual treatments for objects and avatars
- First shipped on PC due to VR rendering constraints
Environment filters highlight fixed geometry; outlines distinguish interactive objects and avatars.
Audio Cues & Descriptions
Spatial audio layer for nonvisual navigation
- Ambient environmental audio grounding users in the scene
- Collision cues when bumping into boundaries
- Automatic spoken descriptions using speech synthesis in MRTK
- Core principle: anything with a visual cue should also have an audio cue
Spatial audio, collision cues, and spoken descriptions provide a nonvisual layer of information.
VR Magnifier
A magnifier tool built for use in VR.
- Allows users to zoom into distant or detailed content
- Groundbreaking accessibility feature for virtual reality
- Critical for low-vision users in immersive environments
The magnifier brings distant content into a readable view.
While these features were found to be helpful for users who wanted to immerse themselves and naturally explore the 3D worlds, not everyone wanted to wander around 3D spaces manually searching for content.
Users Want Structured Control
“I want to quickly scan through all the content and hear information read out to me — like on a website.”A recurring request across user studies
Across studies, this request came up repeatedly.
This insight drove the creation of a new interaction model.
Structured Navigation for 3D
Mesh environments contain multiple rooms, elements, avatars, and objects — with no inherent focus order. To make them nonvisually explorable, we needed to invent one.
3D Scene Reading
I designed a UI system that converts complex 3D scenes into simple 2D overlays using familiar controls (lists, tabs, menus) and existing keyboard navigation patterns.
Choose a Place
in the world
Explore People or Things
within that Place
Select from Actions
available for interaction
A structured interface for discovering places, people, things, and available actions.
Why It Works
- Reduces cognitive load — one Place at a time
- Uses familiar patterns — like navigating a website
- Enables intentional interaction — users choose what matters
- Screen-reader compatible — structured labels and descriptions
MRTK Integration: Accessibility patterns integrated into Mixed Reality Toolkit, extending inclusive design to Unity developers.
Shaping the Future of Accessible Immersive Technology
First-of-its-kind Features
Shipped groundbreaking accessibility features enabling blind and low-vision users to navigate mixed reality
Design Patent Filed
A design patent was filed for the structured 3D interaction model.
Internal Recognition
Recognized as the leading expert on spatial accessibility within Microsoft
Cross-Org Collaboration
Partnered with Xbox, Windows, and Microsoft Legal to form early spatial accessibility guidelines
MRTK Integration
Integrated accessibility patterns into Mixed Reality Toolkit, extending inclusive design to Unity developers
Industry Standards
Collaborated with Meta and Unity on system-level affordances, helping set early industry standards for accessible 3D environments
This work established foundational patterns for spatial accessibility, demonstrating that immersive technology can and should be designed for everyone from the start — not as an afterthought.