MidCase studyProduct strategy
Weekly retention dropped 10%. How do you investigate?
Try answering first
Model answer
I start by confirming the drop is real, then narrow down where it comes from.
- Validate the data. Check for tracking changes, bugs in the pipeline, a changed definition, or seasonality such as holidays.
- Locate it. Segment by cohort (signup week), platform and app version, country, acquisition channel, plan, and user type. A drop concentrated in one segment points to a specific cause.
- Compare with changes. Line it up against releases, experiments, pricing changes, marketing campaigns, outages, and competitor moves.
- Find the stage. Check whether new-user retention fell (an onboarding or acquisition quality issue) or existing-user retention fell (a product or value issue), and which behavior stopped.
- Form hypotheses and test. Combine funnel data with session replays and user interviews, then run a fix or rollback and watch the metric.
I would communicate early with stakeholders, share what is known and unknown, and set up an alert so the next drop is caught sooner.
Key points
- Rule out data and definition problems first
- Segment by cohort, platform, channel and version to localize the cause
- Correlate with releases, experiments and external events
- Combine quantitative data with qualitative research
Likely follow-ups
- What if the drop only appears in one acquisition channel?
- How would you decide between rolling back and fixing forward?
Common mistakes
- Assuming a product problem before checking the data
- Looking only at the overall number without segmenting