
<FullImage src="/landingimgs/tap_landing.png" width={2208} height={1104} priority />

-- Learners were quietly quitting a VR training platform. I designed the AI layer that brought them back.

TAP3D prepares workers for advanced manufacturing roles across VR and Web. As the sole product designer on a six-person team, I took an AI agent from risky idea to funded roadmap — by simulating it with 24 real users before a single model was wired up.

<StatGrid
  stats={[
    { value: "+38%", label: "Issue resolution rate", sub: "33% → 71% in the WoZ study" },
    { value: "+25%", label: "Task start rate", sub: "38% → 63% in the WoZ study" },
    { value: "+33%", label: "Next-step acceptance", sub: "25% → 58% in the WoZ study" },
    { value: "6 mo", label: "End-to-end ownership", sub: "sole designer, 6 partners" },
  ]}
/>

<Team
  teammates={[
    { name: "Jiho Kim", position: "Lead Product Designer" },
    { name: "Product Manager", position: "Roadmap & Constraints" },
    { name: "Engineers", position: "Platform & VR" },
    { name: "QA", position: "Edge Cases" },
  ]}
/>

<ColorShifter background="rgb(255, 255, 255)" color="rgb(26, 26, 26)" />

## What shipped

<LayoutMainContent grid="2/3" extraMargin>
  <Image src="/TAPimgs/AI.png" width={800} height={600}>
    <Caption
      label="1"
      title="A proactive agent"
      text="Instead of a passive course list, the AI opens with one bite-sized suggestion and a reason why — “5 mins, prevents setup errors in Module 2.”"
      overlay={false}
      topPadding
    />
  </Image>
  <Image src="/TAPimgs/Web_interface.png" width={1200} height={800}>
    <Caption
      label="2"
      title="The weekly report"
      text="Progress, earned credit, and one concrete next step — context that used to evaporate between sessions."
      overlay={false}
      topPadding
    />
  </Image>
</LayoutMainContent>

<LayoutMainContent extraMargin>
  <Image src="/TAPimgs/TAP3D_Final_Product.png" width={2639} height={1810}>
    <Caption
      label="3"
      title="Live across the training flow"
      text="The agent shipped as a modular design-system component — consistent from web dashboard to VR checklist."
      overlay={false}
      topPadding
    />
  </Image>
</LayoutMainContent>

None of this existed six months earlier. Here's how it got funded.

<SectionTitle num="01" kicker="The Problem" title="Silent churn, module by module" />

Internal data revealed a significant lag in the training completion cycle — directly impacting customer retention metrics.

<Bars
  bars={[
    { label: "Module 1", value: 92, note: "nearly everyone starts" },
    { label: "Module 3", value: 41, note: "the cliff", highlight: true },
    { label: "Review", value: 12, note: "almost nobody arrives" },
  ]}
  caption="Training completion, monthly cohort analysis. Values are illustrative — actual data is confidential."
/>

The quantitative data showed where learners dropped off. It couldn't show why.

<SectionTitle num="02" kicker="Research" title="9 interviews. 3 barriers." />

I conducted nine in-depth interviews and affinity-mapped 200+ coded quotes around two questions: what triggers a user to start right now, and what's the minimum information needed to commit?

<LayoutMainContent extraMargin>
  <Image src="/TAPimgs/Tap3D_affinity_mapping.png" width={2330} height={1630}>
    <Caption
      label="4"
      title="Affinity mapping"
      text="Synthesizing 200+ quotes from 9 interviews into recurring patterns."
      overlay={false}
      topPadding
    />
  </Image>
</LayoutMainContent>

<Barriers
  items={[
    {
      label: "Barrier 01",
      title: "Commitment debt",
      quote: "It feels too long.",
      text: "A full lecture reads as a heavy burden, so learners defer starting at all.",
      solution: "Bite-sized checklist — 5–10 minute actions instead of lectures.",
    },
    {
      label: "Barrier 02",
      title: "Decision paralysis",
      quote: "I don't know what to do first.",
      text: "Too many options, no ranking — so the safest move is inaction.",
      solution: "One personalized suggestion with a one-line rationale.",
    },
    {
      label: "Barrier 03",
      title: "Support gap",
      quote: "There's no quick help.",
      text: "Stuck learners defer to “tomorrow” — and tomorrow rarely comes.",
      solution: "Context-aware handoff — a rage quit becomes a seamless escalation.",
    },
  ]}
/>

<Insight>
  Learners weren't lazy — starting felt expensive. Every barrier was a cost the design could remove.
</Insight>

<MethodNote
  title="How the barriers were found and ranked"
  items={[
    { label: "Sample", text: "9 in-depth interviews with active learners — scripts co-planned with the PM, who also scheduled participant access." },
    { label: "Synthesis", text: "200+ quotes coded and affinity-mapped; findings presented at weekly engineering syncs to align on problem framing before proposing any solutions." },
    { label: "Ranking", text: "Barriers prioritized jointly with the PM by severity × frequency — that ranking is what scoped the MVP down to a checklist." },
  ]}
/>

<SectionTitle num="03" kicker="Strategy" title="Ship fast, prove more later" />

I evaluated three options on an impact-versus-feasibility matrix:

<TradeOff
  options={[
    {
      label: "Option 1",
      title: "Human mentor",
      text: "High trust, handles complex issues.",
      verdict: "Low availability, high cost, slow.",
    },
    {
      label: "MVP",
      title: "Checklist",
      text: "Low dev cost, leverages existing LMS metadata.",
      verdict: "Quick delivery, immediate value.",
      selected: true,
    },
    {
      label: "Option 3",
      title: "AI agent",
      text: "Personalized, 24/7, context-aware.",
      verdict: "High risk — needs validation first.",
    },
  ]}
/>

The AI agent clearly won on impact — but engineering was mid-flight on AI inside the VR sessions, with latency and token costs unresolved. So I proposed a phased path:

<Flow
  nodes={[
    { label: "Phase 1 — Now", title: "Checklist MVP", note: "low cost, quick win" },
    { label: "Validate", title: "Wizard of Oz simulation", note: "prove ROI with n = 24" },
    { label: "Phase 2 — Ship", title: "AI Agent", note: "full rollout", state: "result" },
  ]}
/>

<LayoutMainContent extraMargin>
  <Image src="/TAPimgs/MVP_Checklist.png" width={2560} height={1810}>
    <Caption
      label="5"
      title="Phase 1 — the Checklist Card"
      text="Pinned to the training page to clarify tasks and reduce cognitive load. Shipped in weeks, not months."
      overlay={false}
      topPadding
    />
  </Image>
</LayoutMainContent>

<SectionTitle num="04" kicker="Validation" title="So I became the AI" />

The MVP solved "see the task list" — but not "what should I do first," "I'm stuck," or "what did I do last week." The agent could solve all three, but it existed only in mockups, and mockups don't win budget conversations.

For a **Wizard of Oz simulation with 24 users**, I played the agent's backend in real time — reading uploads, drafting handoffs, recommending next steps — while the team observed real behavior with a product that didn't exist yet.

<Flow
  nodes={[
    { label: "The learner", title: "Uses the “AI agent”", note: "real tasks on the real platform — believes it's live" },
    { label: "Behind the curtain", title: "Me, operating in real time", note: "reading uploads, drafting handoffs, picking next steps", state: "result" },
    { label: "The observers", title: "PM + QA, logging behavior", note: "conversion events, hesitations, failure patterns" },
  ]}
  caption="The WoZ setup — a working AI experience, no AI required."
/>

Playing the backend meant every reply the "AI" sent was typed by hand — which is how the recommendation script got tuned line by line. The pattern that survived: one action, one reason, one timebox.

<Transcript
  messages={[
    { from: "Learner", text: "I have so much to do, but I don't know what to do first.", tone: "bad" },
    { from: "“AI” — me, backstage", text: "Here's a 5-min safety check — it prevents the setup errors that stall people in Module 2. Do this one now.", tone: "good" },
  ]}
  caption="A WoZ exchange, recreated from session notes. Suggestions without the one-line rationale got ignored; with it, they got started."
/>

<Placeholder
  label="Fig — WoZ session in progress"
  note="A photo from a simulation session: participant on the platform, operator station out of view. Anonymize participant faces."
  aspect="16/9"
/>

<List>
  <ListItem label="33→71%">Issue handoff — upload a recording, AI drafts the instructor message. Under 2 minutes vs. 24+ hours.</ListItem>
  <ListItem label="38→63%">Task start rate — one recommended 5–10 minute action with a one-line rationale.</ListItem>
  <ListItem label="25→58%">Next-step acceptance — weekly summaries pair progress with one concrete suggestion.</ListItem>
</List>

<MethodNote
  title="How the simulation was run"
  items={[
    { label: "Setup", text: "24 learners used the “agent” on the real platform, doing real training tasks, believing it was live. Sessions ran in rounds so fixes could ship between them." },
    { label: "Operator", text: "Me, backstage in real time — reading uploaded recordings, drafting instructor handoffs, picking each next-step recommendation by hand." },
    { label: "Observers", text: "The PM watched sessions live, logging conversion events and hesitations; QA flagged failure patterns invisible from the operator seat." },
    { label: "Measured", text: "Conversion on the three target flows — issue handoff, task start, next-step acceptance — compared against the same flows without the agent." },
  ]}
/>

### Decision 01 — The sort order that buried the product

Mid-simulation, the task-start numbers stalled in a strange way: users were adding checklist items but never starting them. Watching sessions live with the PM, we found the culprit — the list sorted oldest-first, an LMS default nobody had questioned, so the task a learner just added landed at the bottom. The core loop — add a task, do the task — was being defeated by a sort order.

<TradeOff
  options={[
    {
      label: "As tested",
      title: "Oldest-first",
      text: "Chronological, the platform default — finish what you started first.",
      verdict: "Users added items, then never started them. Anything new sat buried below the fold.",
    },
    {
      label: "Between rounds",
      title: "Newest-first + “New”",
      text: "The task you just added is the first thing you see, flagged with a “New” indicator.",
      verdict: "Task starts jumped immediately in the next round — part of the 38% → 63% lift.",
      selected: true,
    },
  ]}
/>

This is the argument for simulating with rounds instead of a one-shot test: the flaw was caught, fixed, and re-validated inside the same study — before a line of production code existed.

<LayoutMainContent extraMargin>
  <Image src="/TAPimgs/TAP3D_checklist_UI.png" width={2136} height={516}>
    <Caption
      label="6"
      title="The fix, shipped between rounds"
      text="Newest-first sorting with a “new” indicator — the change that came straight out of watching WoZ session behavior, not a critique."
      overlay={false}
      topPadding
    />
  </Image>
</LayoutMainContent>

<SectionTitle num="05" kicker="Outcome" title="The CFO said yes" />

I presented AI Agent screens side-by-side against MVP screens to Engineering, the PM, and the CFO. The CFO's concern was ROI and development cost.

<Insight label="The turning point">
  Numbers from a simulation did what mockups couldn't — they made the ROI conversation concrete.
</Insight>

<BeforeAfter
  before="/TAPimgs/MVP_Checklist.png"
  after="/TAPimgs/TAP3D_Final_Product.png"
  width={2560}
  height={1810}
  beforeLabel="MVP — Checklist"
  afterLabel="AI Agent"
  caption="The comparison I presented — the checklist MVP against the AI Agent that replaced it."
/>

We rolled out the agent immediately after the VR AI integration wrapped. I then pitched that a consistent AI experience creates job-ready talent efficiently — and that localizing for workforce-demand markets is key to scale. The CFO agreed; **Chinese localization is now in progress**.

<LayoutMainContent extraMargin>
  <Image src="/TAPimgs/TAP_case_hero_pic.png" width={4943} height={3206}>
    <Caption
      label="7"
      title="One system, every surface"
      text="The agent as it rolled out — the same AI layer across the web dashboard, weekly reports, and the VR headset."
      overlay={false}
      topPadding
    />
  </Image>
</LayoutMainContent>

<Placeholder
  label="Fig — CN localization in progress"
  note="Side-by-side of the AI widget in English and Chinese, showing the CJK reflow guidelines in action."
  aspect="2/1"
/>

<SectionTitle num="06" kicker="Collaboration" title="One designer, six partners" />

As the sole designer, collaboration wasn't a phase — it was embedded in every step:

<List>
  <ListItem label="Wk 1–3">Research — co-planned interview scripts with the PM; weekly affinity-map syncs with engineering.</ListItem>
  <ListItem label="Wk 4">Strategy — engineering flagged latency and token costs early, shaping the phased rollout.</ListItem>
  <ListItem label="Wk 5–8">Design — bi-weekly Figma critiques; QA reviewed empty states, errors, and network failures.</ListItem>
  <ListItem label="Wk 9–12">Validation — the PM observed WoZ sessions live and helped catch the sort-order issue.</ListItem>
  <ListItem label="Wk 13–24">Ship & scale — co-built the reusable AI widget; defined CN localization and CJK reflow rules.</ListItem>
</List>

<SectionTitle num="07" kicker="Reflection" title="What I'd carry forward" />

-- Simulation testing seeded a culture of data-driven design at TAP3D.

Pre-launch simulation wasn't common there before this project — proving its value to the C-suite changed how the company validates ideas. It also taught me that a phased MVP isn't a compromise; it's the path that makes the ambitious version fundable.

<LayoutMainContent extraMargin>
  <Image src="/TAPimgs/TAP3D_Zoom.png" width={2014} height={999}>
    <Caption
      label="8"
      title="The pitch"
      text="Presenting WoZ simulation results to the C-suite for AI rollout approval."
      overlay={false}
      topPadding
    />
  </Image>
</LayoutMainContent>

<Spacer />



<FullImage src="/landingimgs/lepal_landing.png" width={1084} height={1084} priority />

-- Most wellness apps speak. Few listen. We built one that does.

As Lead Product Designer at Lepal.ai, I designed an emotionally intelligent journaling companion from 0→1 — shifting the AI from efficiency to empathy to break a critical 72-hour retention wall for Gen Z users.

<StatGrid
  stats={[
    { value: "+15%", label: "D7 retention lift", sub: "variant cohort" },
    { value: "+85%", label: "“Supportive” perception", sub: "post-test survey" },
    { value: "24→60%", label: "User satisfaction", sub: "control → variant" },
    { value: "30→45%", label: "Engagement", sub: "control → variant" },
  ]}
/>

<Team
  teammates={[
    { name: "Jiho Kim", position: "Lead Product Designer" },
    { name: "Product Manager", position: "Roadmap & Metrics" },
    { name: "Engineers ×2", position: "Platform" },
    { name: "ML Engineer", position: "Emotion Classifier" },
    { name: "Conversation Designer", position: "Tone Copy" },
    { name: "CTO", position: "Architecture" },
  ]}
/>

<ColorShifter background="rgb(245, 240, 255)" color="rgb(20, 10, 40)" />

## What shipped

<LayoutMainContent grid="1/1" extraMargin>
  <Image src="/lepal_img/Frame 2087325946.png" width={1790} height={1008}>
    <Caption
      label="1"
      title="My Journal"
      text="A mood-aware journaling flow where the AI reads the room before it responds — the focus of this study."
      overlay={false}
      topPadding
    />
  </Image>
  <Image src="/lepal_img/Frame 2087325947.png" width={1790} height={1008}>
    <Caption
      label="2"
      title="A companion, not a chatbot"
      text="Your spirit levels up with you — daily rituals like Crystal Ball keep the relationship alive between journal entries."
      overlay={false}
      topPadding
    />
  </Image>
</LayoutMainContent>

<LayoutMainContent extraMargin>
  <Image src="/lepal_img/case_study_hero.png" width={2411} height={994}>
    <Caption
      label="3"
      title="The full ecosystem"
      text="Journaling, daily micro-interactions, and Therapy Planet — topic-driven AI chat for deep dives into burnout or relationships."
      overlay={false}
      topPadding
    />
  </Image>
</LayoutMainContent>

While I designed the entire ecosystem, the biggest challenge lay in My Journal, where the AI needed to feel like a supportive companion, not a robot. Getting there is the story of this case study.

<SectionTitle num="01" kicker="The Problem" title="The 72-hour wall" />

Despite a promising beta, data showed strong Day-1 interest but rapid disengagement around the 72-hour mark.

<Bars
  bars={[
    { label: "Day 1 — Excited", value: 100, note: "“It actually responded nicely”" },
    { label: "Day 2 — Lukewarm", value: 68, note: "“Kinda generic”" },
    { label: "Day 3 — The Wall", value: 31, note: "empathy mismatch", highlight: true },
    { label: "Day 7 — Gone", value: 14 },
  ]}
  caption="Active users by day — a 40% spike in session drop-offs at the wall."
/>

This is the exchange that was killing us, over and over, on Day 3:

<Transcript
  messages={[
    { from: "User, Day 3", text: "I'm wiped after back-to-back labs. Zero energy." },
    { from: "AI", text: "Got it! Let's do a deep reflection with 5 questions to boost productivity!", tone: "bad" },
  ]}
  caption="The drop-off trigger. The user reached out exhausted — and got homework."
/>

The issue wasn't app performance. The AI was **failing to read the room**. And the team was split on the root cause:

<TradeOff
  options={[
    {
      label: "Engineering",
      title: "“It's a memory bug.”",
      text: "Backend wasn't persisting context, causing disjointed conversations.",
      verdict: "Fix session persistence",
    },
    {
      label: "Design (me)",
      title: "“It's an empathy gap.”",
      text: "The AI remembered facts but failed to read feelings. Users didn't feel heard.",
      verdict: "Redesign tone & interactions",
      selected: true,
    },
  ]}
/>

Both stories fit the data. Only one of them could be tested with users.

<SectionTitle num="02" kicker="Research" title="Listening to 800 sessions" />

I ran a mixed-method study: five semi-structured interviews with Gen Z graduate students and a 3-day mood diary, analyzing 1,000+ quotes across 800+ sessions. Three patterns kept repeating:

<Barriers
  items={[
    {
      label: "Finding 01",
      title: "The empathy gap",
      quote: "It remembers my exam. Not how I felt about it.",
      text: "The AI retained facts but never read feelings.",
      solution: "Emotion classifier — four mood states, read from context.",
    },
    {
      label: "Finding 02",
      title: "Generic tone",
      quote: "I said I was drained. It gave me productivity tips.",
      text: "One default register for every emotional state.",
      solution: "Adaptive tone rules — pacing and structure match the mood.",
    },
    {
      label: "Finding 03",
      title: "Blank-page freeze",
      quote: "I don't know what I'm supposed to write.",
      text: "An empty chat box reads as a test, not an invitation.",
      solution: "Hybrid starters — guided first run, clean canvas after.",
    },
  ]}
/>

<Insight>
  Users felt unseen. The AI was offering solutions when they needed validation.
</Insight>

A backend fix wasn't enough — we needed an emotional overhaul.

<Placeholder
  label="Fig — Research synthesis, recreated"
  note="A sanitized recreation of the affinity map or mood-diary synthesis (original artifacts are under NDA). Even an abstracted version — clusters, counts, three finding labels — carries the section."
  aspect="16/9"
/>

<SectionTitle num="03" kicker="Strategy" title="Teaching the AI to read the room" />

I reframed the goal from "fixing memory" to "prioritizing empathy before solutions," and collaborated with ML engineers and the PM on an emotion-aware framework: instead of one-size-fits-all replies, the system classifies every input into one of four emotional states — anxious, sad, neutral, positive — and adapts its tone, structure, and pacing to match.

<FullImage
  src="/lepal_img/The Adaptive Tone Framework — how the system classifies emotion and adjusts responses.png"
  width={3585}
  height={1000}
  caption="The Adaptive Tone Framework — how the system classifies emotion and adjusts responses"
/>

<Flow
  nodes={[
    { label: "Input", title: "“Just got back from meeting my advisor.”", note: "reads neutral to a keyword filter" },
    { label: "Context", title: "LLM checks emotional history", note: "recent entries show high anxiety" },
    { label: "Classified", title: "Anxious", note: "one of four core states" },
    { label: "Response", title: "Calm & Validating", note: "acknowledge → tiny step → soft close", state: "result" },
  ]}
  caption="Context over keywords — the same sentence gets a different response depending on the user's emotional history."
/>

Each state carries its own response spec — not just word choice, but structure and pacing. This is what the conversation designer and I wrote templates against:

<List>
  <ListItem label="Anxious">Calm & Validating — acknowledge → tiny step → soft close. Short sentences, minimal exclamations.</ListItem>
  <ListItem label="Sad">Warm & Gentle — sit with the feeling before offering anything.</ListItem>
  <ListItem label="Neutral">Curious & Open — invite more, don't assume a mood.</ListItem>
  <ListItem label="Positive">Celebratory & Affirming — match the energy, reinforce the win.</ListItem>
</List>

Here's the framework itself — pick a message and watch it read the room. Notice the pause: heavier emotions get a longer, more deliberate "thinking" delay before the reply.

<ToneDemo />

<SectionTitle num="04" kicker="Design" title="Two decisions that made it feel human" />

### Decision 01 — The thinking indicator

Even with an empathetic tone, instant responses felt robotic. We needed a cue that implied care, not loading.

<TradeOff
  options={[
    {
      label: "A",
      title: "Generic indicator",
      text: "“Generating response...” — clear system status, easy to build.",
      verdict: "Too functional. Feels like waiting for a server, not a friend.",
    },
    {
      label: "B",
      title: "Typing dots",
      text: "“Thinking...” — mimics human behavior.",
      verdict: "Feels like a companion is thinking about what you said.",
      selected: true,
    },
  ]}
/>

I shipped the dots with variable latency — 1–1.5s for light check-ins, up to 2–3s for heavy emotional sharing — shifting perception from "the app is downloading a reply" to "someone is thinking about me."

<BeforeAfter
  before="/lepal_img/lepal_before.png"
  after="/lepal_img/lepal_after.png"
  width={620}
  height={465}
  beforeLabel="Before"
  afterLabel="After"
  caption="Generic instant response vs. the tone-adaptive thinking indicator with variable latency."
/>

### Decision 02 — The blank page dilemma

Users froze at an empty chat box, but permanent FAQ guidance clutters the chat and biases input. The hybrid: contextual starters on first run only ("My head is a mess from labs and assignments 🤯"), fading out once a user engages, with a discreet "i" button as the safety net.

<LayoutMainContent extraMargin>
  <Image src="/lepal_img/lepal_after2.png" width={1240} height={466}>
    <Caption
      label="4"
      title="Guided, then out of the way"
      text="New users get onboarding, returning users get a clean canvas — and the Tone Framework gets authentic data."
      overlay={false}
      topPadding
    />
  </Image>
</LayoutMainContent>

Neither decision was mine to ship alone — both meant convincing people whose instincts said the opposite.

<SectionTitle num="05" kicker="Collaboration" title="“Latency isn't lag — it's listening”" />

When I proposed variable latency, the lead engineer pushed back: **"You want us to add artificial latency?"** The line above was my answer — but it only landed because we tested it. Against an instant-reply control, the 1.5–3s variant scored significantly higher on "feeling heard" with **zero complaints about speed**. Variable latency shipped as a feature, not a bug.

<MethodNote
  title="How the latency claim was tested"
  items={[
    { label: "Ceiling", text: "Set before testing: intentional delay builds trust, excessive lag breaks it — so latency was capped at a 3-second maximum, scaled by emotional weight below that." },
    { label: "Conditions", text: "Control: instant replies under 100ms — the engineering position. Variant: 1.5–3s variable latency with the thinking indicator." },
    { label: "Measured", text: "Post-session “feeling heard” score, plus any complaint mentioning speed. The variant scored significantly higher; speed complaints: zero." },
  ]}
/>

The same pattern resolved the "friction vs. data" debate with the PM: starters on for new users, off for returning ones, FAQ button as the safety net. Retention protected, data quality intact.

The Tone Framework became the team's shared language — design owned tone rules, engineering owned latency logic, the PM owned metrics, the conversation designer owned empathy markers.

<Placeholder
  label="Fig — The framework as shared artifact"
  note="The Tone Framework doc/FigJam annotated with each discipline's ownership area — or a workshop photo. This is the collaboration proof-shot."
  aspect="16/9"
/>

<Quote who="Lead Engineer" title="Lepal.ai">
  I didn't see it as a UX problem until she reframed latency as listening.
</Quote>

<SectionTitle num="06" kicker="Outcome" title="From productivity tool to companion" />

We ran a pre-launch A/B test: the original productivity-driven tone as control, the emotion-aware framework as variant. The variant won on every measure — retention, engagement, satisfaction, and how "supportive" users rated the AI — and shipped in the launched mobile app.

<MethodNote
  title="How the A/B test was designed"
  items={[
    { label: "Conditions", text: "Same product, one difference: the control kept the original productivity-driven tone; the variant ran the full emotion-aware framework — classifier, tone rules, variable latency." },
    { label: "Measured", text: "D7 retention on the variant cohort (+15%), engagement (30% → 45%), satisfaction (24% → 60%), and “supportive” perception from a post-test survey (+85%)." },
    { label: "Ownership", text: "The PM owned success metrics and test design — the same shared-framework split that ran through the whole project." },
  ]}
/>

<Placeholder
  label="Fig — Control vs. variant"
  note="The two test conditions side by side: the same user message answered in productivity tone vs. emotion-aware tone, with the survey deltas."
  aspect="2/1"
/>

<SectionTitle num="07" kicker="Reflection" title="What I learned" />

-- Good product design is about designing relationships.

I aligned Engineering's need for logic with Design's need for empathy by building a shared framework they could both contribute to — conflict became co-creation. And success wasn't about adding more tools: a smaller, warmer feature set won over a larger, colder one.

The best AI products don't just process language — they understand context, respect emotion, and create space for human vulnerability.

<Spacer />



<FullImage src="/landingimgs/sidewalk_landing.png" width={2208} height={1104} priority />

-- An accessibility tool that wasn't accessible. I rebuilt its foundations.

Project Sidewalk is an open-source civic platform that crowdsources urban accessibility data to train machine learning models. With over 350,000 labels collected, it had proven its value — but the architecture and experience underneath were brittle. I architected the zero-defect, WCAG 2.2 AA design system now adopted across three cross-functional teams.

<StatGrid
  stats={[
    { value: "350k+", label: "Labels crowdsourced" },
    { value: "100%", label: "WCAG 2.2 AA compliance", sub: "zero axe-core violations" },
    { value: "3", label: "Teams adopted the system" },
    { value: "11k+", label: "Users supported" },
  ]}
/>

<Team
  teammates={[
    { name: "Jiho Kim", position: "Product Designer, System Architecture" },
    { name: "Product Manager", position: "Rollout & Prioritization" },
    { name: "Developers ×2", position: "Frontend Integration" },
  ]}
/>

<ColorShifter background="rgb(240, 253, 244)" color="rgb(10, 30, 15)" />

## What shipped

<LayoutMainContent extraMargin>
  <Image src="/ProjectSidewalk/PS_Mockup.png" width={4340} height={2860}>
    <Caption
      label="1"
      title="The redesigned platform"
      text="Clearer labeling flows and navigation, in production for volunteers across 8+ cities."
      overlay={false}
      topPadding
    />
  </Image>
</LayoutMainContent>

<LayoutMainContent extraMargin>
  <Image src="/ProjectSidewalk/PS_FinalPRoduct.png" width={3456} height={2234}>
    <Caption
      label="2"
      title="Map navigation, rebuilt"
      text="Optimized pathfinding for the core exploration experience — where 68% of clicks happen."
      overlay={false}
      topPadding
    />
  </Image>
</LayoutMainContent>

The screens are the visible part. The real deliverable is the system underneath them — and the reason it had to exist is where this story starts.

<SectionTitle num="01" kicker="The Problem" title="An accessibility tool that failed WCAG" />

I identified three critical friction points holding the platform back:

<Barriers
  items={[
    {
      label: "Friction 01",
      title: "User drop-off",
      text: "Inconsistent UI and unclear feedback caused high abandonment in core reporting flows.",
      solution: "One selection model and feedback pattern across every flow.",
    },
    {
      label: "Friction 02",
      title: "Engineering bottleneck",
      text: "No centralized system — developers rebuilt UI from scratch, shipping regression bugs.",
      solution: "A reusable component library treated as a working contract.",
    },
    {
      label: "Friction 03",
      title: "The accessibility irony",
      text: "A tool for accessibility that failed WCAG itself — alienating the community it serves.",
      solution: "AA compliance baked into every component's foundation.",
    },
  ]}
/>

An initial WCAG 2.2 AA audit made the debt concrete:

<Bars
  bars={[
    { label: "Color contrast", value: 37, display: "37% pass" },
    { label: "Keyboard navigation", value: 42, display: "42% pass" },
    { label: "Screen reader", value: 28, display: "28% pass", note: "weakest area", highlight: true },
    { label: "Focus indicators", value: 51, display: "51% pass" },
  ]}
  caption="Initial WCAG 2.2 AA assessment across core pages. Values are illustrative — actual audit data is confidential."
/>

<Insight>
  The tool built to map accessibility barriers had become one.
</Insight>

<SectionTitle num="02" kicker="Alignment" title="Three user groups, one system" />

Redesigning a civic platform meant balancing competing needs simultaneously:

<List>
  <ListItem label="Mappers">Citizens & volunteers — label barriers quickly, low cognitive load, compliant interface.</ListItem>
  <ListItem label="Consumers">City planners & governments — data accuracy, standardized reporting, error-proof UI.</ListItem>
  <ListItem label="Internal">Engineers & maintainers — scale to 8+ cities without regressions, fewer UI bugs.</ListItem>
</List>

A zero-defect design system was the only way to satisfy all three at once.

<SectionTitle num="03" kicker="Process" title="Design as infrastructure" />

<Flow
  nodes={[
    { label: "1 — Audit", title: "Data-informed diagnostics", note: "Hotjar · community voice · WCAG audit" },
    { label: "2 — Build", title: "Zero-defect architecture", note: "tokens → component library" },
    { label: "3 — Scale", title: "Cross-functional adoption", note: "phased rollout across 3 teams", state: "result" },
  ]}
/>

### 01 — Diagnose with data, not intuition

Hotjar heatmaps and session recordings pinpointed drop-off moments; GitHub issues and forum threads surfaced recurring pain; a heuristic WCAG evaluation scored the legacy interface. The behavioral data was decisive — **68% of clicks concentrated on the map center** while labels outside the viewport went undiscovered, and **42% of users abandoned the sidebar controls** after one attempt.

<LayoutMainContent extraMargin>
  <Image src="/ProjectSidewalk/mockup-map-labeling.png" width={1950} height={1230}>
    <Caption
      label="3"
      title="The labeling interface, rethought"
      text="Filters, validation states, and severity in one sidebar; label cards and the live map share one selection model."
      overlay={false}
      topPadding
    />
  </Image>
</LayoutMainContent>

### 02 — Tokens as a single source of truth

I mapped Figma variables 1:1 with the engineers' SCSS variables — `color.primary` to `$color-primary`, `spacing.md` to `$spacing-md` — eliminating guesswork during handoff.

<LayoutMainContent extraMargin>
  <Image src="/ProjectSidewalk/PS_designsystem_token.webp" width={4096} height={1826}>
    <Caption
      label="4"
      title="Design ↔ SCSS parity"
      text="Every token exists identically in design and code — one source of truth, zero ambiguity."
      overlay={false}
      topPadding
    />
  </Image>
</LayoutMainContent>

On that foundation I built the component library with **WCAG 2.2 AA baked in** — any future feature built from these components inherently complies. Accessibility stopped being a task and became a property of the system. Here's the validation-button system, alive:

<SystemDemo />

<LayoutMainContent extraMargin>
  <Image src="/ProjectSidewalk/PS_designsystem_overview.webp" width={1400} height={1212}>
    <Caption
      label="5"
      title="The component library"
      text="Validation buttons in 4 states and 2 sizes, 9 semantic label tokens, 4 brand primitives."
      overlay={false}
      topPadding
    />
  </Image>
</LayoutMainContent>

<LayoutMainContent extraMargin>
  <Image src="/ProjectSidewalk/mockup-component-library.png" width={2480} height={1240}>
    <Caption
      label="6"
      title="Living documentation"
      text="Every component page pairs the preview with usage, accessibility notes, and the exact tokens — AA compliance is checked per variant."
      overlay={false}
      topPadding
    />
  </Image>
</LayoutMainContent>

### 03 — A working contract, not documentation

A design system is only as successful as its adoption rate. Working tightly with one PM and two developers, I ran phased rollouts with comprehensive spec handoffs — treating the system as a contract between design and engineering rather than a style guide nobody reads.

<SectionTitle num="04" kicker="Impact" title="What zero-defect bought us" />

<LayoutMainContent extraMargin>
  <Image src="/ProjectSidewalk/mockup-wcag-dashboard.png" width={2480} height={1200}>
    <Caption
      label="7"
      title="The audit, passing"
      text="48/48 criteria, 34/34 components, zero violations — six legacy issues resolved, none remaining."
      overlay={false}
      topPadding
    />
  </Image>
</LayoutMainContent>

<List>
  <ListItem label="11k+">Users supported — clearer navigation significantly improved contributor task completion.</ListItem>
  <ListItem label="100%">WCAG 2.2 AA — zero violations in automated axe-core audits across all 24 components.</ListItem>
  <ListItem label="3 teams">Adopted the system — faster delivery, and UI-related frontend bugs eliminated.</ListItem>
</List>

<SectionTitle num="05" kicker="Reflection" title="Systems thinking in civic tech" />

-- A design system is not a style guide — it's a tool for empowerment.

This project shifted my perspective from designing for the happy path to designing for resilience in open-source and civic environments. When inclusivity lives in the system architecture, every new feature inherits those values — accessibility becomes automatic instead of an afterthought.

<Spacer />



<FullImage src="/landingimgs/cheffy_landing.png" width={1084} height={1084} priority />

-- When voice and gestures fail, the screen adapts to your elbow.

Recipe video apps assume clean, dry fingers. In reality, users are elbow-deep in flour, raw meat, or wet ingredients. As Founding Designer leading interaction and AI prototyping, I designed a multi-signal adaptive UI that detects when cooks can't use standard inputs — and switches itself to elbow-operable targets.

<StatGrid
  stats={[
    { value: "−91%", label: "Hand-wash interruptions", sub: "6.2 → 0.5 per recipe" },
    { value: "−78%", label: "Step transition time", sub: "15.2s → 3.3s" },
    { value: "97%", label: "Elbow-tap success rate", sub: "18-user controlled study" },
    { value: "+34%", label: "Recipe completion", sub: "500-user beta · 4 weeks" },
  ]}
/>

<Team
  teammates={[
    { name: "Jiho Kim", position: "Founding Designer — Interaction & AI Prototyping" },
    { name: "Tools", position: "Figma · ProtoPie · TensorFlow Lite · Swift" },
  ]}
/>

<ColorShifter background="rgb(255, 247, 237)" color="rgb(30, 20, 10)" />

## What shipped — try it

The product itself is confidential (stealth mode), so here's the core interaction, rebuilt for this page. Lean in:

<ElbowDemo />

That switch — from standard controls to one elbow-sized target in 400ms — is the whole product. The rest of this study is how we got there.

<SectionTitle num="01" kicker="The Problem" title="Wash, dry, tap, repeat" />

72% of cooking-video users pause to wash their hands before tapping "Next." Every interaction costs a wash → dry → tap → resume cycle — **12–18 seconds per step transition**. And the obvious fallback fails exactly when it's needed most:

<Transcript
  messages={[
    { from: "Cook, hands in flour", text: "Hey, next step" },
    { from: "Voice assistant", text: "Sorry, I didn't catch that. Try again?", tone: "bad" },
  ]}
  caption="Range hood at 72dB — 22dB past the point where voice recognition holds up."
/>

<Flow
  nodes={[
    { label: "Voice", title: "“Hey, next step”", note: "failed · 72dB ambient noise", state: "fail" },
    { label: "Gesture", title: "Waves flour-covered hand", note: "confidence 34% · occluded", state: "fail" },
    { label: "Manual", title: "Stop → wash → dry → tap", note: "+15s wasted", state: "result" },
  ]}
  caption="The input failure cascade — every fallback ends in a hand-wash."
/>

I audited NYT Cooking, Tasty, and Paprika — the failure is structural, not a missing feature:

<Barriers
  items={[
    {
      label: "Audit 01",
      title: "Touch fails",
      text: "Tiny arrows and swipe carousels — precise micro-interactions become a physical barrier for coated hands.",
      solution: "One elbow-scale macro target instead of precision taps.",
    },
    {
      label: "Audit 02",
      title: "Voice fails",
      text: "“Next step” breaks against sizzling pans, running water, and blenders — high error rates, high frustration.",
      solution: "Treat voice failure as a signal, not a dead end.",
    },
    {
      label: "The gap",
      title: "Nobody designs for the mess",
      text: "Users need control without clean hands or a silent room — and nothing on the market offers it.",
      solution: "An interface that adapts itself when inputs degrade.",
    },
  ]}
/>

<SectionTitle num="02" kicker="Research" title="36 hours in twelve kitchens" />

<List>
  <ListItem label="Inquiry">12 home cooks observed in-kitchen across 3-hour sessions — 36 hours recorded.</ListItem>
  <ListItem label="Sensors">Proximity and ambient noise across cooking phases — 2,400+ data points.</ListItem>
  <ListItem label="Diary">7-day logging of "moments I wanted to tap but couldn't" — 84 entries.</ListItem>
</List>

<Bars
  bars={[
    { label: "Idle", value: 35, display: "35dB" },
    { label: "Prep", value: 45, display: "45dB" },
    { label: "Chopping", value: 55, display: "55dB" },
    { label: "Sauté", value: 65, display: "65dB" },
    { label: "Range hood on", value: 72, display: "72dB", note: "voice already failing", highlight: true },
    { label: "Frying", value: 80, display: "80dB" },
  ]}
  caption="Kitchen noise by cooking phase. Voice recognition degrades past ~50dB — most of active cooking happens above it."
/>

<LayoutMainContent extraMargin>
  <Video src="/project-assets/cheffy/cheffy-kitchen-observation.mp4" width={1344} height={768} seekOnScroll={false}>
    <Caption
      label="Field"
      title="Mid-recipe, hands unusable"
      text="Recreated from the contextual-inquiry sessions: recipe propped on the counter, hands coated, the next step one impossible tap away."
      overlay={false}
      topPadding
    />
  </Video>
</LayoutMainContent>

Voice failed most during the noisiest phases — exactly when hands-free control mattered most. Gesture confidence collapsed with flour, mitts, or utensils in hand. But the most surprising finding was behavioral: **nine of twelve participants** were already trying to tap with their elbows. The 44px targets were just too small for it to work.

<Insight>
  The users had already designed the solution. We just had to build an interface big enough for it.
</Insight>

<SectionTitle num="03" kicker="The System" title="An interface that listens to failure" />

The design responds to failure instead of demanding new behavior. A multi-signal AND-gate watches three conditions — only when all three trip does the UI switch:

<Flow
  nodes={[
    { label: "Trigger 1", title: "Voice input failed", state: "fail" },
    { label: "Trigger 2", title: "Gesture confidence < 40%", state: "fail" },
    { label: "Sensor", title: "Proximity < 30cm", state: "pass" },
    { label: "State change", title: "Elbow-Bump Mode", note: "bottom 50% becomes one giant target", state: "result" },
  ]}
  separator="+"
  caption="The AND-gate — two failed inputs plus an approaching user. Any single signal alone changes nothing."
/>

### Decision 01 — Why three signals, not one

The first prototype switched modes on proximity alone — and it was unusable. Leaning in to read a step, checking the pan, reaching past the tablet: nearly one in four activations was false, and every false switch taught users to distrust the mode. It took three prototype rounds to earn the gate:

<TradeOff
  options={[
    {
      label: "Round 1",
      title: "Proximity alone",
      text: "Switch the moment the user comes within 30cm — simplest to build, instant response.",
      verdict: "23% false activations. The UI flipped every time someone leaned in just to read.",
    },
    {
      label: "Round 3",
      title: "The AND-gate",
      text: "Voice failed + gesture confidence below 40% + proximity under 30cm. All three, or nothing.",
      verdict: "Under 3% false activations. The UI changes only when the user demonstrably can't use standard inputs.",
      selected: true,
    },
  ]}
/>

The gate is conservative on purpose: a missed activation costs one hand-wash, but a false one costs trust in the whole mode.

### Decision 02 — Sizing for a body part nobody designs for

Touch guidelines assume an 8–12mm fingertip. An elbow lands as a 40–60mm blob, with far less aim — so before sizing anything, we measured real elbow contact areas (4.2 × 6.8cm) and forearm approach angles. That's why the target is the entire bottom half of the screen rather than a bigger button: roughly 120× the area of a standard 44px target, reachable without looking.

Two more decisions rounded out the system:

<List>
  <ListItem label="Overlay">An 85% gradient keeps the recipe visible beneath the button — users never lose their place.</ListItem>
  <ListItem label="Auto-revert">8 seconds without proximity reverts to standard controls; double-tap force-exits.</ListItem>
</List>

<Placeholder
  label="Fig — The trigger logic across three prototype rounds"
  note="Three frames side by side: what triggered the switch in each round, and what changed after watching it fail. Cropped ProtoPie frames or Figma snapshots work — this is the process proof-shot."
  aspect="2/1"
/>

<SectionTitle num="04" kicker="Validation" title="Testing with flour-covered hands" />

A controlled study with 18 participants cooking real recipes: hand-wash interruptions fell from 6.2 to 0.5 per recipe, transitions dropped from 15.2s to 3.3s, and elbow taps landed 97% of the time.

<MethodNote
  title="How the study was designed"
  items={[
    { label: "Design", text: "Within-subjects: each of the 18 participants cooked the same 8-step recipe twice — once on the standard UI, once with adaptive mode." },
    { label: "Control", text: "Current best practice: standard 44px touch targets, voice commands, wash-to-tap fallback." },
    { label: "Measured", text: "Hand-wash interruptions per recipe, step-transition time, and elbow-tap success on the macro target." },
  ]}
/>

<LayoutMainContent extraMargin>
  <Video src="/project-assets/cheffy/cheffy-elbow-bump.mp4" width={1344} height={768} seekOnScroll={false}>
    <Caption
      label="Mid-test"
      title="One target, no washing"
      text="Recreated from the usability sessions — dough-covered hands and the oversized target that takes a bump instead of a fingertip."
      overlay={false}
      topPadding
    />
  </Video>
</LayoutMainContent>

<Quote who="P7" title="Home cook, 3 years experience">
  I literally forgot I was using my elbow after the second step.
</Quote>

<Quote who="P12" title="Cooking instructor">
  The big button appearing when I got close was like the phone reading my mind.
</Quote>

<SectionTitle num="05" kicker="Outcome" title="From concept to shipped feature" />

After three rounds of prototyping, the adaptive mode shipped to a 500-user beta over four weeks. Removing the hand-wash friction didn't just save time — it changed how people finished recipes:

<List>
  <ListItem label="+34%">Recipe completion rate.</ListItem>
  <ListItem label="+22%">Session duration.</ListItem>
  <ListItem label="+41%">7-day return rate.</ListItem>
  <ListItem label="97.2%">Trigger accuracy, with mode-switch latency under 400ms.</ListItem>
</List>

<MethodNote
  title="Where the beta numbers come from"
  items={[
    { label: "Rollout", text: "500-user beta group tracked over four weeks, after the third prototype round." },
    { label: "Accuracy", text: "97.2% trigger accuracy against a 2.8% false-activation rate — logged automatically from live mode-switch events during the beta." },
    { label: "Latency", text: "Default-to-macro transition under 400ms — fast enough that users read it as the phone anticipating them, not reacting." },
  ]}
/>

<SectionTitle num="06" kicker="Reflection" title="Failure is an input" />

-- The real breakthrough came from listening to what's already failing.

<TradeOff
  options={[
    {
      label: "Old approach",
      title: "Add more input modalities",
      text: "Stack voice, gesture, gaze, touch — hope one works.",
      verdict: "Each new layer adds latency, false positives, and user confusion.",
    },
    {
      label: "New approach",
      title: "Listen to failures, then adapt",
      text: "Failure signals are the input. Two failed modalities plus proximity means the system already knows what to do.",
      verdict: "Zero new learning curve.",
      selected: true,
    },
  ]}
/>

The pattern isn't kitchen-specific. A living room where the remote is lost and voice is misheard, an industrial floor where gloves block touch and machinery kills voice — any environment where hands, voice, or attention are compromised benefits from the same AND-gate logic.

-- Context is the best input. When everything else fails, the system already knows what to do — that's now my default.

<Spacer />



-- AR that shows you where to grab — published research helping people with low vision cook safely and independently.

CookAR is a head-mounted AR system that recognizes kitchen tools in real time and overlays affordance augmentations — visual cues that distinguish the part of a tool you grab from the part that can hurt you. It received the Best Paper Award (Belonging and Inclusion) at UIST 2024, ACM's flagship venue for user interface software and technology.

<StatGrid
  stats={[
    { value: "ACM", label: "UIST 2024 Best Paper", sub: "Belonging and Inclusion Award" },
    { value: "10", label: "Low-vision participants", sub: "qualitative lab study" },
    { value: "1st", label: "Egocentric dataset", sub: "of kitchen tool affordances" },
    { value: "9", label: "Co-authors", sub: "UW · UW–Madison · UT Dallas" },
  ]}
/>

<LayoutMainContent extraMargin>
  <Image src="/Sub_proj/CookAR_gif.webp" width={330} height={330}>
    <Caption>CookAR in action — real-time affordance augmentations rendered on kitchen tools through a head-mounted AR display.</Caption>
  </Image>
</LayoutMainContent>

The system pairs a fine-tuned affordance segmentation model with a stereo-camera AR headset, generating real-time visual augmentations directly on the tools a user is reaching for. The team collected and annotated the first egocentric dataset of kitchen tool affordances to train it, then refined the augmentation designs through a lab study with ten low-vision participants.

I co-authored the research as part of a cross-institution team spanning the University of Washington, the University of Wisconsin–Madison, and UT Dallas.

[Read the paper on the ACM Digital Library ↗](https://dl.acm.org/doi/10.1145/3654777.3676449)


I'm Jiho, a product designer. I design AI products that adapt to people,and prototype them in code .

From passive lists to proactive agents

I'm Jiho, a product designer. I design AI products Hover meTap methat adapt to people,and prototype them in code .

From passive lists to proactive agents

I'm Jiho, a product designer. I design AI products that adapt to people,and prototype them in code .