Context / Problem
Users Were Dropping Off — But Why?
TAP3D is a B2B VR training platform preparing workers for advanced manufacturing roles, offering a seamless experience between VR and Web.
Despite a seamless platform, internal data revealed a significant lag in the training completion cycle. Users were struggling to complete training and move to the review stage, directly impacting customer retention metrics.
Expected
Complete Training
Goal
Higher Retention
Reality
Early Drop-off
Training Completion Rate
Monthly cohort analysis · Illustrative
Started
Drop-off point
Module 1
Module 2
Module 3 ⚠
Module 4
Module 5
Review
Sharp drop-off at Module 3
— data showed where, but not why
* Values are illustrative. Actual data is confidential.
Usage logs showed a clear pattern — learners were dropping off early, unable to keep pace with initial modules. The quantitative data showed where they dropped off, but not why.
Onboarding
100%
Initial Modules
Struggling
Drop-off
← Churn Risk
Review Stage
Unreached
Research
9 Interviews. 3 Barriers.
To uncover the root cause, I conducted 9 user interviews and performed affinity mapping on 200+ coded quotes to identify recurring themes.
My research centered on two key questions: What triggers a user to start right now? and What is the minimum information needed to commit?
User Interviews
9 in-depth interviews with learners
Affinity Mapping
200+ coded quotes mapped for recurring themes
Affinity mapping — synthesizing 200+ quotes from 9 user interviews into recurring patterns
9
user interviews conducted
200+
coded quotes analyzed via affinity mapping
3
critical cognitive barriers confirmed
Research confirmed three critical barriers that explained the drop-off:
Barrier
Commitment Debt
"Feels too long." Learners deferred because a "full lecture" felt like a heavy burden.
Solution
Bite-sized Checklist
AI recommends 5–10 min actions instead of full lectures, making starting feel effortless.
Barrier
Decision Paralysis
"I have so much to do, but I don't know what to do first."
Solution
Personalized Suggestion
AI picks the best next step with a one-line rationale — "Here's a 5-min safety check. Do this now."
Barrier
Support Gap
"No quick help." Without an immediate way to get answers, learners would defer tasks to "tomorrow."
Solution
Context-Aware Handoff
Users upload a recording → AI auto-drafts a message with context for the instructor. A "rage quit" becomes a seamless escalation.
Core Goals
Make starting feel bite-sized. Make "why today" obvious. Provide a quick help path.
Strategy
Ship Fast, Prove More Later
I evaluated three options to address these goals, considering development cost and user value.
Option 1
Human Mentor
High trust, handles complex issues.
Low availability, high cost, slow
MVP ✓
Checklist
Low dev cost, leverages existing LMS metadata.
Quick delivery, immediate value
Option 3
AI Agent
Personalized, 24/7, context-aware.
High risk, needs validation first
↑ High Impact
← Low Feasibility
High Impact
Low Feasibility
Low Feasibility
High Impact
High Feasibility
High Feasibility
Low Impact
Low Feasibility
Low Feasibility
Low Impact
High Feasibility
High Feasibility
Human Mentor
Checklist
AI Agent
Human Mentor
Checklist
AI Agent
High Feasibility →
↓ Low Impact
Human Mentor
Checklist
AI Agent
1
Phase 1 — Now
Checklist MVP
Low cost, quick win
2
Validate
WoZ Simulation
Prove ROI, n=24
3
Phase 2 — Ship
AI Agent
Full rollout
The Phased Approach
Trade-off
The AI Agent clearly won on impact, but engineering was constrained — they were finalizing AI inside the VR sessions. Latency and token costs were concerns.
Decision
Checklist as MVP to deliver immediate value. I designed a "Checklist Card" pinned to the top of the training page to clarify tasks and reduce cognitive load.
MVP Checklist — the first iteration shipped to validate the core training flow
Validation
Simulating AI to Prove ROI
What the MVP couldn't solve
MVP — Checklist
See task list
✓ solved"What should I do first?"
unsolved"I'm stuck, need help now"
unsolved"What did I do last week?"
unsolvedProposed — AI Layer
See task list
Personalized "do this now"
Context-aware issue handoff
Weekly progress summary
To justify the investment and convince skeptical stakeholders, I led a Wizard of Oz simulation with 24 users — I acted as the AI backend in real-time.
Wizard of Oz Simulation
n=24 users · I acted as the AI backend · Real behavior data
01
Issue Handoff — Reducing Friction
Learners upload a short web/VR recording; the system auto-drafts a message with context for the instructor.
Try it — Click "Upload" to experience the handoff
Learner stuck on Module 3
Recording: 0:42s
screen-recording-module3.webm
AI is analyzing the recording...
Extracting context...
AI Draft Ready — Auto-sent to instructor
To: Instructor Kim · Module 3
Resolved in under 2 min — vs. 24hr+ without AI
33%
Before
71%
+38%
02
Personalized Suggestion — Iterative Design
AI recommends a 5–10 minute action with a one-line rationale.
Real-time Iteration
Users added items but didn't start — "Oldest-First" hid new tasks. I iterated to "Newest-First" with a "New" indicator.
Before — Oldest First
Jan 3
Jan 5
Jan 8
New ↓ buried
After — Newest First
New ✨
Jan 8
Jan 5
Jan 3
03
Report Summary — Context Retention
AI summarizes weekly/monthly progress and proposes the immediate next step to sustain motivation.
Your Weekly Summary
Modules completed
3 / 5
Time invested
2h 15m
Safety score
↑ 72% → 88%
Suggested Next Step
Complete "Equipment Lockout" quiz (8 min) to finish your safety certification this week.
Outcome
The CFO Said Yes
I presented a side-by-side comparison (AI Agent screens vs. MVP screens) to Engineering, the PM, and the CFO.
The CFO was initially worried about ROI and development costs. The specific conversion metrics from the simulation test provided concrete evidence. We decided to roll out the AI Agent immediately after the VR AI integration wrapped.
MVP — Checklist
AI Agent
The AI Agent is now live across the training flow. I designed the AI widget as a modular component to ensure consistency across the design system.
I pitched that a consistent AI experience creates job-ready talent efficiently, and that localizing for workforce-demand markets is key to scale. The CFO agreed, and we are now progressing with Chinese localization.
AI Agent Live
Across training flow
Modular Component
Part of design system
Global Expansion
Chinese localization in progress
Post-rollout completion metrics fall outside my engagement window — the validation data above is what secured the investment.
Reflection
What I'd Carry Forward
Pre-launch simulation testing wasn't common at TAP3D. This project demonstrated its value to the C-suite, helping seed a culture of data-driven design.
Validation Culture — Simulation testing proved its value, seeding a culture of data-driven design and early validation at TAP3D.
Cross-Functional Leadership — Navigated constraints with PMs and Engineers, making strategic trade-offs to arrive at the optimal phased solution.
This project taught me to navigate constraints and make strategic trade-offs. I realized I want to design services that scale globally and solve complex ecosystem problems — leveraging cross-disciplinary collaboration and large-scale problem solving.
Pitching WoZ simulation results to C-suite for AI rollout approval