Your dashboard is lying to you — and which three numbers actually tell the truth

Every founder I talk to shows me the same dashboard. MRR going up. Users going up. Sessions going up. They're proud of it. And I get it — the chart looks like progress.

Then I ask one question: "What's your Week 4 retention on the cohort that signed up in January?"

Silence. They don't know. Most dashboards aren't built to show you that. And that's the problem.

The dashboard looks good. The product isn't.

Vanity metrics are seductive because they always go up. Total users never decreases. Pageviews never crash. Cumulative anything is designed to trend right. But as Amplitude's research on vanity metrics puts it, a metric is only valuable if you can act on it to improve your business — and if it keeps climbing regardless of what you do, it's telling you nothing.

Harvard Business Review identified a related failure mode they call "surrogation": companies mentally replace their actual strategy with the metrics meant to measure it. The result is that you optimize the number, not the outcome. HBR found this pattern destroys value — and they have the case studies to prove it, starting with Wells Fargo opening 3.5 million accounts nobody asked for.

You don't need a banking scandal to feel the cost. You need two quarters of chasing the wrong line on a chart.

Here are the three numbers that replace the noise.

Number one: retention by cohort, not aggregate

Aggregate retention is the average of all users who are active right now divided by all users you've ever had. It masks everything. A product losing 60% of each monthly cohort can still show flat or rising aggregate retention if you're acquiring fast enough.

Cohort retention forces honesty. You take every user who signed up in a given week, and you track what percentage of them came back at Week 1, Week 4, Week 8, Week 12. Then you stack those cohorts and look for the shape.

Lenny Rachitsky's analysis on retention benchmarks puts the floor for a good SaaS business at 65%+ revenue retention after year one. The median subscription product retains around 42%. If you don't know your cohort curve, you don't know which side of that line you're on.

Mixpanel's cohort methodology adds one more thing worth watching: whether newer cohorts are retaining better than older ones. If they are, your product is improving. If they aren't, your acquisition channel is masking a product problem.

Number two: activation rate

Activation rate is the percentage of new users who reach the moment where the product actually works for them — the aha moment. Not the sign-up. Not the first login. The specific action that predicts whether they'll come back.

Amplitude calls this the leading metric for retention. Find it by correlating early user actions with Week 4 retention, then tracking what percentage of each signup cohort hits that action within the first 72 hours.

The number itself matters less than the trend. If your activation rate is falling, you have a funnel problem. If it's stable but retention is dropping, you have a product problem. Those are different diagnoses with different fixes. Conflating them is expensive.

This is where I typically loop in Leo (Growth Strategist) — once we know the activation threshold, he runs the experiments to move it.

Number three: revenue per active user

MRR is a lagging indicator. It tells you what happened. Revenue per active user tells you what's happening.

Take your MRR and divide it by users who were active in the last 30 days. Not all users. Not all paying users. Active users. If that number is rising, you're either monetizing better or losing the users who weren't paying anyway — both are good. If it's falling, you're growing headcount with users who generate nothing.

This metric also exposes the difference between a growth story and a dilution story. A product with 1,000 active users at $50 RPAU is a different business than one with 10,000 active users at $4 RPAU, even if the MRR looks similar. Investors who know what they're doing ask this. You should know the answer before they do.

For this number to be trustworthy, the underlying event data has to be clean — which is why I work closely with Nora (Data Architect) before I'll stand behind any RPAU figure. A metric is only as good as the schema it's built on.

What to do with all of this

You don't need to throw out your current dashboard. You need to add three views to it:

1. A cohort retention grid — signup week on the Y axis, time-since-signup on the X axis, retention percentage in each cell.
2. An activation funnel — what the aha moment is, and what percentage of each week's signups hit it within 72 hours.
3. A revenue-per-active-user trend — monthly, last 6 months, broken out by plan tier.

That's it. One afternoon of setup in whatever analytics tool you're already using. You'll know more in the first week of looking at it than you learned from a year of watching MRR.

Show me one week of your data. I'll show you which week to bet on.