In-Product Learning: Educating Users in the Flow of Work

Role

Product Design Lead

Company

MongoDB

Timeline

Q3-Q4 2024

Overview

Goals

Teach highly technical users what they need, when they need it, without slowing them down or driving them out of the product entirely.

Help MongoDB users understand technical concepts and make better configuration decisions when setting up their database and other MognoDB offerings. Support users without bringing them out of the product (i.e. linking out to documentation) as we know that leads to drop-offs in the conversion funnel. Establish patterns and guidelines to be used across the product organization.

Core Team

One Principal Technologist and me. And many, many, stakeholders

Right now the education journey is very disjointed. We build features without thinking about the entire user journey.

senior product manager

Historically haven’t done a good job at in product learning send users straight to docs to or rely on support teams.

Staff Product manager

Problem

Business Context & User Problem

Developers use an overwhelming number of tools every day, and MongoDB's own product surfaces are highly technical. Configuring an index, setting up a Time Series Collection, or choosing the right sharding strategy all require domain knowledge many users don't have on hand. When that knowledge gap isn’t addressed in the product, users leave their flow of work to find the answer, sometimes never returning to complete the task.

Constraints

Human memory and attention spans: Users forget roughly 70% of what they're taught within a day (Ebbinghaus Forgetting Curve), and on an average web page, users read at most 28% of the words during a visit. This meant every word of in-product guidance had to earn its place.

Existing help content decayed quickly: In-product education content that isn't actively maintained becomes wrong as the product evolves, actively confusing users rather than helping them.

The project was explicitly not about driving traffic or conversion: I scoped this deliberately as a non-goal. This was about task-contextual learning (helping someone understand what they're doing right now), not top-of-funnel education or increasing engagement with a specific CTA.

Foundational Research

Stakeholder Interviews

I ran interviews with 15 Product Managers across MongoDB which surfaced a consistent theme: users struggle to pick the right configuration for their use case, and need support for advanced settings, troubleshooting, and best practices. This pattern came up independently, repeatedly, across teams that had never coordinated with each other.

Through these interviews I was able to hi-light the most pressing issues for each team. Cross-referencing these insights with our company priorities allowed me to hone in on our most important use cases for this work.

Design Studio

I ran 3 design studios with product designers from various teams to understand what explorations had already been done, and gather ideas for solutions.

Site Audit and Heuristic Evaluation

The Principal Technologist and I ran an audit and heuristic evaluation of the key flows identified through our stakeholder interviews and by MongoDB leadership. We kept track of our biggest hurdles and used these as areas to redesign or experiment. We also created a document that was disseminated to relevant product designers

Existing Research Review

I conducted a review of prior, related research across other teams that had explored the space. Three other product teams' research independently found that in-product documentation was highly valued specifically when it didn't require leaving the product. This research validated our need to standardize a pattern for in-product learning.

finding 1

Users need support for many key workflows

Users need support for many key workflows

This includes: Creating indexes, understanding timeseries collections and understanding terminology.

finding 2

UX Copy was one of the biggest issues

Show users too much information and it's overwhelming. Show them irrelevant information and they become distracted. Give users too little information and they might struggle to complete their task.

finding 3

Design Patterns were needed

Sometimes how information is presented to users can be just as important as what that information is.

We needed to explore new ways to present contextual information to users to aid them in their tasks without overwhelming them.

finding 4

AI Support is the next phase

AI presents a unique opportunity to help our users, especially in cases that are hard to anticipate.

AI can lower the barrier to entry for certain complex and technical tasks. We should be thinking about how we can leverage AI to simplify tasks.

Research

Usability Study

The Principal Technologist and I ran an extensive usability study to understand which patterns best supported our users in understanding complex concepts.

6 MongoDB users
2 hours each
7 design concepts across 3 scenarios

Experimentation

I collaborated with the Growth product team and the Vector Search team to run an experiment to validate a side panel which would guide

See full research readout

Solutions

Adopt an expandable side panel as the primary in-product learning pattern, which can be easily transitioned to an AI assistant

Users need contextual reference information without permanently occupying screen real estate or forcing a context switch.

This was the clear favorite across test participants: predictable, discoverable, and didn't require abandoning the page to get an answer.

Prioritize on-page, inline content over links to external documentation

Every external link is a chance for a user to get distracted, lose their task context, or simply not come back.

Trade-off: This requires more design and content effort per flow than simply linking out to existing Docs page. We set up a process with the documentation team to own these sections and update as the product is updated.

Use plain, benefit-focused language over MongoDB-specific technical jargon

Complex terms decreased user confidence, even when the surrounding explanation was otherwise clear. I codified this into formal UX writing guidance: focus on user benefit rather than mechanism

Advanced users may want precise technical information which can be included using progressive disclosure (tooltips, expandable descriptions).

Impact

Strategic Impact

Successfully established a cross-product vision and guidance for contextual learning patterns. This included a user-testing-backed set of specific UI patterns

MongoDB Search Experiment

The MongoDB Search product team ran an experiment using our support panel and found users in the treatment group were 20% more likely to run a search query within 2 days of creating their search index

UX Copy Style Guide

Helped identify the need for a UX copy style guide to facilitate clear, standardized copy within our products. Read more about this project.

Setup Milestones Experiment

The MongoDB Growth team ran an experiment using our support panel to explain database setup milestones. It showed lifts of +10.12% and +10.97%, (both statistically significant). However it did not produce meaningful results further down the funnel.

reflection

Waiting for AI

While this project was successful in the short term, the timing was suboptimal given the speed at which AI was developing. Many of our concepts became outdated quickly as AI offered a more scalable solution. We were thoughtful in our implementation, knowing that many of the components we created would soon be updated to incorporate some level of AI support.