WhoBuilt

How to Measure Your Product

Sep 12, 2018· 59 min· 150K views·Watch on YouTube

Speaker

Suhail Doshi, Mixpanel co-founder, explains how to measure product understanding, onboarding, and retention at a Y Combinator talk.

Start with a few growth metrics

Doshi frames measurement around whether people understand a product, can get started easily, and come back to use it. He maps growth to five steps: visits, signups, valuable actions, retention, and sharing. He recommends choosing three to five guiding metrics because large dashboards can confuse teams and stall decisions.

Measure whether people understand

Doshi says a confusing landing page can send visitors straight to the Back button. Mixpanel initially used the tagline “Metrics That’ll Make You Drool” and tried to reach both startups and large companies. Customer conversations and iteration helped the company focus its message on measuring in-product actions, which distinguished it from Google Analytics. Signups and clicks on key links, such as pricing, can indicate interest.

Make the first experience easy

Doshi recommends measuring a funnel from landing page to signup to a first valuable action, and checking how long it takes users to get started. He uses Google’s immediate search experience and Airbnb’s booking flow as examples of different levels of complexity. He advises startups not to add friction such as email confirmation before they need it, and to keep improving onboarding.

Watch retention and churn

Doshi warns that viral growth can hide weak retention, creating a “shark fin” pattern as users leave faster than new ones arrive. He recommends tracking new-user retention over longer periods, daily active users, and monthly dollar churn for subscription businesses. At 7 percent monthly revenue churn, a business loses about 58 percent of its revenue over a year, making growth harder to sustain.

Learn from users and focus

For teams with fewer than 50 users, Doshi says direct conversations can reveal more than charts alone. When Mixpanel could not run a useful A/B test on funnel layouts, 10 of 11 customers preferred a horizontal design, so the team built it. He recommends choosing one North Star metric for six months, reviewing it, and adding only a few other measures to monitor.

Takeaways

  • Track the journey from visit to signup, first valuable action, retention, and sharing.
  • Use signups and key page clicks as early signals of whether your product message is clear.
  • Measure how many users reach a valuable action and how long onboarding takes.
  • Track retention over time and calculate the effect of monthly dollar churn across a year.
  • With a small user base, talk to customers and commit to one guiding metric before expanding the dashboard.

Mentioned in this talk

Search inside this talk

What was claimed 9

  • founded
    “I started this company called Mixpanel.”
    YouTube
    Verified2 sources· Talk, Web· archived
  • metric
    Mixpanel: About 7,000 paying customers and around $100 million in annual recurring revenue
    “we have about 7,000 paying customers. We’re starting to close in on around $100 million in annual recurring revenue”
    YouTube
    Unverified1 source· Talk
  • feature
    Mixpanel: Measures what people do inside a product, including funnels and retention.
    “We help you measure what people are doing inside your product.”
    YouTube
    Unverified1 source· Talk
  • feature
    Airbnb: Homepage invites users to search for unique homes and experiences as the start of the booking flow.
    “Their homepage has a simple message: ‘Book unique homes and experiences.’”
    YouTube
    Unverified1 source· Talk
  • feature
    LivingSocial: Its early Facebook app went viral but was not retentive, prompting a company pivot.
    “LivingSocial was basically an app on Facebook ... It went super viral.”
    YouTube
    Unverified1 source· Talk
  • metric
    LinkedIn: The company once cited 200 million registered users.
    “LinkedIn saying, seven or eight years ago, that it had 200 million registered users.”
    YouTube
    Unverified1 source· Talk
  • metric
    Mixpanel: Daily active use correlated with churn; a small increase in daily active use could reduce churn by around 10 percent.
    “Even a small increase in daily active use could reduce churn by around 10 percent”
    YouTube
    Unverified1 source· Talk
  • metric
    Slack: About 80 percent retention, described as unusually high.
    “Something like Slack would probably have 80 percent retention”
    YouTube
    Unverified1 source· Talk
  • metric
    Mixpanel: Early first-year retention was around 30 to 40 percent.
    “when we first started out, had 30 to 40 percent retention in the first year”
    YouTube
    Unverified1 source· Talk

Related talks