This work is password protected.

Enter the password to view this case study. Don't have it? Reach out via LinkedIn.

← Back to portfolio

UX Research Case Study

When Reporting Fails,
Customers Leave.

A discovery-to-validation research program that surfaced the root causes of CRM churn, and directly shaped design, product and business decisions.

Role Research Lead & Strategy
Collaborators Product, UX, Customer Success, Support, Engineering, Data Science, Marketing
Duration 8 sprints ~ 16 weeks
Methods Desk Research · Stakeholder Interviews · Ethnography · In-Depth Customer Interviews · Moderated Usability · Rapid Iteration
44%
of churned CRM customers
cited reporting as the reason
15
customers interviewed
400+
existing reports in the system
customers regularly used only 5–7
01
Context & Problem Space

Internal churn data flagged reporting as a leading cause of customer cancellations.

But the team lacked a clear picture of who was affected, why, and what "better reporting" looked like.

The Business Problem
Customers were exporting raw CRM data, rebuilding reports in Excel, and running the same report dozens of times across individual stores because the CRM didn't support their workflow.
📊
The data existed, but couldn't be trusted
Customers wanted insight from their CRM data, but inconsistencies between reports eroded confidence. Many kept parallel spreadsheets just to verify what the CRM told them.
📂
Too many reports, too little clarity
Over 400 reports existed in the system. Dealers ignored most of them, not because they weren't useful, but because there was no way to find or organize the ones they needed.
⚡
A market opportunity and a retention risk
Competitors were gaining ground with cleaner dashboards and self-service reporting. Fixing this wasn't just about retention, it was about competitive positioning.
♡
The Team

Three humans. One shared goal.

Great research doesn't happen in isolation. This project was a genuine collaboration, research, design, and product working in close partnership from first question to final pilot.

Kamala Alcantara
Kamala Alcantara
Senior UX Researcher
Research Driver

Designed and led the research program end-to-end, from stakeholder tours and dealer interviews to synthesis, recommendations, workshop facilitation, and stakeholder alignment across the org.

Research StrategyInterviewsSynthesisFacilitationRecommendationsStakeholder Alignment
Leban Hyde
Leban Hyde
Senior UX Designer
Design Lead

Translated research insights into tangible product direction — leading problem framing, desk research, user flow mapping, wireframing, and prototyping through to a proof of concept ready for pilot.

Problem FramingUser FlowsWireframingPrototypingPOC Design
Mary Swan
Mary Swan
Product Manager
Product Connector

The connective tissue between research, design, and engineering — keeping the team aligned on priorities, ensuring the right features got built, and championing the user's voice through every product decision.

RoadmapPrioritizationCross-functional AlignmentStakeholder Management
02
Research Process

From discovery to validation.

This project followed a mature, multi-phase research program: grounding in data before touching a single user, surfacing real workflow needs, then validating a designed solution.

Phase 1 — Foundation
Desk Research & Stakeholder Immersion
Started with internal churn reports, amalyzing Amplitude usage data, and customer satisfaction signals to understand the scale and shape of the problem. Then conducted a structured stakeholder tour: Product Manager, Design Manager, Product Marketing, Customer Success Managers (CSMs), Regional Managers, Sales, Business Managers and the Director of Engineering and his team to build a 360° picture of what the team knew and where the gaps were. This phase ensured every subsequent research question was grounded in both business reality and user signal.
Phase 2 — Discovery
Ethnography · 5 Local Dealerships · 20+ Users
Visited local dealerships to observe their reporting workflows and rituals mornings, afternoons, and evenings for the length of 2 weeks. This was the real research magic!
Phase 2 — Discovery
Moderated User Interviews · 8 Dealerships · 15 Users
Conducted remote moderated interviews with 15 dealership users across 8 different auto groups, spanning roles from BDC Managers and Sales Managers to GMs, Marketing Leaders, and owners. Sessions followed a structured but conversational moderator guide covering: current reporting habits, pain points and workarounds, customization needs, trust in data, and self-service readiness. Participants represented small independent dealer groups, single-brand medium dealerships, and multi-location non-enterprise auto groups.
Phase 3 — Synthesis
Thematic Analysis & UX Recommendations
Synthesized findings into key insights, three major workaround patterns, role-based customer journey maps, and a prioritized UX opportunity table mapping pain points to business impact (Very High / High / Medium). Deliverables included an executive summary, task performance scorecard framework, role-based insights, and an engaging deck presented to 150+ product, design, customer success, sales and engineering leaders.
Phase 4 — Divergence
Two Parallel Bets: AI Tool vs. Research-Backed UX Design
Research recommendations spawned two development paths. An engineer pursued an AI-powered reporting tool based on a niche customer suggestion. In parallel, the design team built a structured self-service Report Builder directly informed by the research findings. Both paths were tested: one failed fast, one succeeded.
Phase 5 — Validation
Usability Testing of Report Builder · Pilot Preparation
Conducted moderated usability testing of the live Report Builder with original research participants, evaluating 5 core task scenarios. Internal testing passed. The product is now heading to pilot with a cohort of priority dealerships (customers), closing the loop from research to impact.
03
Participants

Real users from real stores.

Participants were recruited from 8 non-enterprise dealerships, representing a range of roles, group sizes, and geographic contexts. Target profiles: Sales Managers, BDC / Internet Managers, General Sales Managers, Marketing Leaders, and CRM Managers.

Name Role Dealership
Participant 01BDC ManagerMulti-Location Non-Enterprise Auto Group
Participant 02Director, BDC SalesMulti-Location Non-Enterprise Auto Group
Participant 03BDC Assistant ManagerMulti-Location Non-Enterprise Auto Group
Participant 04Regional General ManagerSingle-Brand Medium Dealership
Participant 05General Sales ManagerSingle-Brand Medium Dealership
Participant 06Fixed Ops DirectorSingle-Brand Medium Dealership
Participant 07BDC ManagerMulti-Location Non-Enterprise Auto Group
Participant 08BDC DirectorSmall Independent Dealer Group
Participant 09Ecommerce DirectorMulti-Brand Small Dealer Group
Participant 10Owner / Digital Operations LeadSmall Independent Dealer Group
Participant 11Sales / Digital MarketingMulti-Location Non-Enterprise Auto Group
Participant 12CRM ManagerMulti-Location Non-Enterprise Auto Group
Participant 13Sr. Digital Marketing ManagerMulti-Location Non-Enterprise Auto Group
Participant 14Sales ManagerMulti-Location Non-Enterprise Auto Group
Participant 15VP of MarketingSmall Independent Dealer Group
Research Design Note
Targeting 3 primary user archetypes: Sales & BDC Managers, General Sales Managers & Directors, and Marketing Leaders allowed the research to capture how different roles intersect with the same reporting system in fundamentally different ways, surfacing friction that role-agnostic surveys missed.
04
Research Questions

Seven questions that shaped everything.

Each question was designed to map directly to a potential product or design decision.

05
Key Insights

Findings that cut through the noise.

Every insight below was grounded in direct user quotes, behavioral observation, and corroborating patterns across multiple participants and roles.

🔧 Top Workarounds
"95% of the time we take your reports, export them, and redo them."
→ Insight: Dealers want flexibility and control — but inside the product, not in Excel. Limited customization, clunky interfaces, and poor formatting drove rebuilding from scratch.
"I have to pull reports for each store and cut/paste to make a group report."
→ Insight: Multi-store and longitudinal reporting was entirely missing. Dealers needed consolidated cross-store views without repetitive, error-prone manual steps.
"We keep our own Google Doc to track monthly leads because CRM is wrong."
→ Insight: Dealers were building shadow systems to verify CRM data. Trust in data was critically low — an urgent signal for better validation and transparency.
"It takes me 2.5 hours to run a single report across all our stores...just waiting for it to load. And I do this every morning..."
→ Insight: Beyond UX friction, raw performance was a dealbreaker. Reporting was described as a slow, manual process that bullied the entire workday.
💡 Key Insights
📋
Dealers use only 5–7 reports
Despite 400+ report templates in the system, users relied on a predictable handful of 5-7 report templates.
👥
Duplicate leads erode everything
Without built-in duplicate detection, customers spent hours manually exporting and cross-referencing data. Duplicates inflated metrics, confused reps, and made accurate coaching nearly impossible.
🎛️
"Customization" ≠ building from scratch
Users didn't want SQL-level power, they wanted templates they could tweak. Filter by store, hide columns, sort by rep. Sales managers often just wanted to reorder a few fields or combine two standard reports.
⏱️
Workflow inefficiency was the real killer
Reports couldn't be saved, batched, scheduled, or easily refreshed. Every new timeframe or location meant starting over. It was hours of lost time per week, per manager.
🏬
Multi-store reporting didn't exist
Group-level managers had zero native tools for cross-store visibility. They wanted rollups, region filters, and gosh darnit just pull one report for multiple stores.
📉
Data trust was dangerously low
Customers frequently asked cowrokers: "How did you get this number?" when comparing reports. The inconsistency between CRM figures and other sources had normalized distrust as a baseline behavior.
06
Role-Based Pain Points

Same system. Three different experiences.

Customer journey mapping revealed how radically different the reporting experience felt depending on the role — and how each archetype required distinct design solutions.

Sales & BDC Managers
Top Pain Points
  • No way to sort tasks by type or priority; unworked leads aren't flagged
  • No visibility into communications quality (repeated templates, email/call mix)
  • Reports require multiple exports; metrics live in different formats/sources
  • Lead source performance skewed without clean filtering
  • No multi-store or aggregate reporting function
General Sales Managers & Directors
Top Pain Points
  • Long load times; no combined reports across stores
  • No single view of Set–Show–Sold; inconsistent metrics
  • Monthly wrap-up: conflicting data, time-consuming cleanup, duplicate lead detection relies on manual workarounds
  • UX/UI clunky for drilldowns, not enough guidance or filters
Marketing Leaders
Top Pain Points
  • No cross-channel attribution; hard to isolate website-only leads
  • Data spread across multiple systems
  • Manual comparison; no multi-store dashboards; no anomaly alerts
  • CRM data not presentation-ready; can't automate report delivery or annotate key takeaways
07
UX Recommendations

From insight to opportunity — prioritized by impact.

Recommendations were mapped to a three-tier impact framework based on observed user behavior: how many users were affected, whether the issue blocked core tasks, and whether it was directly tied to churn risk.

Opportunity Evidence Impact
Allow bulk report actions across multiple stores Users spent 10–15 hours weekly running the same reports across multiple stores individually Very High
Add familiar UI elements that explain report metrics (hover tooltips, etc.) Dealers frequently asked "How did you get this number?" when comparing conflicting reports — leading to mistrust Very High
Introduce more controls for key reporting filters (date picker, toggles for lead type, role, individuals) Users wanted to filter by "Feb vs. March leads" and simplify overly dense reports Very High
Enable split-deal reporting support Users expressed frustration over not being able to track split deals accurately, slowing work Very High
Enable familiar customization UI (favoriting, drag-and-drop columns) Multiple users described wanting a simple "Lego-like" builder instead of requesting heavy SQL-based reports High
Allow users to discover and interact with reports and search for key metrics Users stated they don't know what reports exist and rely on CSMs to activate them manually High
Add scheduling options for recurring report delivery Dealers asked "Can I have it sent to me once a week?" — but noted it wasn't possible Medium
Improve visual hierarchy and reduce noise in report layouts Dealers described the current UI as "depressing," overloaded, and hard to interpret in meetings Medium
08
What Happened Next

Research sparked two paths. One taught us timing. One shipped.

After the research readout, the team pursued two parallel development bets — each grounded in real signal. Here's what we learned from both.

⚡ Path A — Failed Fast
AI-Powered Reporting Tool

Customers are genuinely open to AI, and that signal came through in our research too. One participant raised the idea of an AI assistant that could help answer reporting questions naturally, and it sparked a real question worth exploring: could AI meaningfully reduce the friction we were seeing?


An engineer ran with it and built a prototype. It was a worthy bet. What we discovered is that the infrastructure wasn't quite ready to support it well — data consistency issues, trust gaps, and workflow complexity meant that AI-assisted reporting introduced new uncertainty on top of existing uncertainty. Dealers needed reliability and control first.


This wasn't a dead end; it was a timing and sequencing insight. The appetite for AI is there. The foundation just needs to get there first. We deprioritized it without overinvesting, and redirected toward what the broader research clearly supported.

What we learned: AI in reporting is a when, not an if. Customers want it; but they need to trust the underlying data before they'll trust an AI interpreting it. Get the foundation right, then layer intelligence on top.
✓ Path B — Heading to Pilot
New Research-Informed Reporting Experience

Directly informed by the research, the design team built Report Builder: a structured self-service reporting experience that addressed the five core findings head-on.


Features shipped: CRM permission-based access, multi-location reporting, base report templates, add/remove/reorder columns, data preview, save and organize reports, save column + filter configurations, CSV/Excel export, and scheduled delivery.


Internal usability testing validated the core workflows. Participants completed high-value tasks independently, expressed increased confidence in data, and — critically — stopped reaching for Excel. The product is now entering pilot with dealerships from the original research cohort.

Research lesson: When the design solution maps directly to validated needs, it earns trust in testing faster.
09
Internal Proof of Concept Workshop · October 2025

Before dealers tested it, our own team did.

Before taking the Report Builder to customers, I designed and facilitated an internal usability lab workshop with CSMs and support staff — the people closest to dealer pain points — to pressure-test the proof of concept and surface issues early.

Why This Step Mattered
Running an internal POC review before a pre-pilot is a deliberate risk-reduction move. Internal participants could roleplay dealer scenarios, flag broken flows, and surface terminology confusion — all without putting the relationship with real dealerships on the line. It also aligned the team (PM, designer, researcher) on what "ready" actually looked like.
Usability Lab: Report Builder POC Workshop FigJam board
⊕ View Workshop Board
👥
Who participated
Internal CSMs and Tier 3 support staff — participants with direct experience translating dealer reporting needs into CRM configurations. They tested using real dealership accounts to simulate authentic usage.
🗓️
Session structure
~60 minute moderated session: 5 min intro, 5 min FigJam note-taking orientation, 35 min core task flow (filters → columns → save → download → schedule), 15 min wrap-up and reflections.
🔬
What we were evaluating
Usability, discoverability, and learnability across the core Report Builder workflow — with specific attention to multi-store filtering, column customization, save/export flows, and scheduled delivery.
Core Task Scenarios Tested
Filters: Start from a base report, adjust date filters, pull prior month data, and compare performance across multiple store locations.

Column Settings: Add a missing column, remove an unneeded one, search for a new column, and reorder columns to match a preferred layout.

Save: Save the customized report — then find it again and make another round of edits. Save as overwrite vs. save as copy.

Download: Export the report to Excel/CSV and validate the output met expectations.

Schedule Delivery: Schedule the report for weekly inbox delivery — evaluating scheduling clarity and trust in automation.
How Feedback Was Captured
Participants added sticky notes directly to a shared FigJam board during the session — organized by task and color-coded by theme. Leban moderated the tasks, I managed the FigJam and notetaking in real time, and Mary observed. This kept synthesis fast and made patterns immediately visible to the whole team without a lag between session and debrief.
10
From Research to Reality

The artifacts that bridged insight and product.

Research doesn't end with a report or presentation. Here's what the work looked like as it moved from findings into design decisions, prototypes, and engineering handoff.

Personas & Workarounds from dealer interviews
⊕ View
Personas & Workarounds
Customer journey maps and workaround patterns from dealer interviews — the research artifacts that shaped every design decision.
Value statement and feature considerations
⊕ View
Designer's Discovery Board
Value statement, target market, user needs, and feature priorities — the designer's translation of research into product direction.
Concept mockups and prototype screens
⊕ View
Concept Mockups
Early prototype screens of the Report Builder UI — research recommendations made tangible, ready for usability testing.
Engineering handoff documentation
⊕ View Full Handoff Doc
Engineering Handoff
The handoff doc that took research-backed designs to the engineering team — closing the loop from user need to shipped feature.
11
Up Next · Usability Validation Study

The product launch team is ready, and so are we!

The next step is bringing original research participants back to test the live Report Builder in Pilot. We've defined five core task scenarios and clear success criteria, ready to run as soon as the pilot begins.

🔜 Coming in Pilot Phase
This study is planned and ready — not yet completed. The design is heading into pilot with dealerships, and this validation study will run alongside it to measure whether the solution truly delivered on the research findings.
Five Planned Task Scenarios
Task 1 — Start From a Base Report: Navigate to Report Builder, locate the Set → Show → Sold report, and adjust date filters for last month.

Task 2 — Customize Columns: Add, remove, and reorder columns to match a presentation-ready view. Evaluate drag-and-drop clarity and column definition comprehension.

Task 3 — Multi-Location Reporting: Generate a report comparing performance across all stores. Evaluate ease of location selection, rollup understanding, and trust in aggregated totals.

Task 4 — Save & Organize: Save the customized report, name it, and locate it again. Evaluate naming clarity and folder/organization model.

Task 5 — Schedule & Export: Schedule the report for weekly delivery and download as CSV. Evaluate scheduling discoverability and export formatting.
📐
Quantitative Metrics Planned
Task success rate, time on task, error rate, assistance required, System Usability Scale (SUS) score, and confidence rating (1–5) will be captured for every session.
🗣️
Qualitative Signals Planned
Thematic coding around trust ("I believe this number" vs. "I'd double check"), customization clarity, efficiency, cognitive load, and terminology confusion.
🎯
Success Criteria
80%+ task completion without moderator assistance. Multi-store reporting completed in 3–5 minutes. Documented less reliance on team (less support tickets), and more!
12
Up Next · Outcomes & Impact

Where this is headed.

The pilot research plan is ready. The build is stable. Data science is working on tagging so we have metrics. The pilot is coming.

🏁
Current Status
Heading to Pilot
🤝
Teams Aligned
PM + Eng + Design
⚡
Failed Fast On
AI Tool — Not Yet
📋
Deliverables
6 Artifacts
🔜 Hypotheses — To Be Validated in Pilot
→ Users will successfully customize base reports without needing support — our internal POC workshop gave us confidence, but pilot testing with real dealers will be the true measure.
→ Multi-store managers will complete cross-store reporting faster than in the legacy workflow — the design directly addresses the 26-export problem. Pilot will tell us by how much.
→ Save + schedule features will be perceived as high value — every dealer we spoke to wanted exactly this. We expect it to be the most celebrated feature in the pilot.
→ Column-level control will reduce Excel dependency — the core promise of the whole product. We'll measure this directly through task completion and post-session reflection.
What This Project Already Shows
Even before pilot results are in, this project demonstrates what rigorous, longitudinal UX research can do when it's treated as a strategic asset, not a checkbox before design starts. A 44% churn signal became a research program. A research program became a product. A product is now heading to the customers who helped build it.
🚗