When metrics are dangerous

When are metrics dangerous? When they lead to making decisions that put your venture in peril.

I was talking with a founder who was enthusiastic about his progress. Revenue up 300% over last year! Team size up by 500%! He committed to raising $1.5M in a seed round with numbers like that.

Then I started to dig deeper. What was last year’s revenue? $100k. What is this year’s expected revenue? $300k. Yes, he tripled his revenue. But is this the signal to invest all of his energies and bet the company on a seed raise?

Photo by Arie Wubben on Unsplash

Metrics are beneficial and also very dangerous. The purpose of metrics is to help sort signals from noise. If you look at the wrong metric, you will miss critical signals. Or worse, interpret false signals that lead you to lose everything.

When revenue is below $1M, the percentage revenue increase is meaningless. Indeed, all metrics at this very early stage are noise because they have no reference and no history.

At this initial stage, the founder is at the very beginning of H1 (Credibility). The focus of H1 is finding problem-solution fit, and specifically, the value innovation. Problem-solution fit focuses on identifying what specific job-to-be-done you propose to solve in a way that adds 10x value to the target user. This means mapping each step of the interaction to see the one subcomponent that is “broken”. You want to understand precisely why a buyer chooses you – what makes your solution more valuable than their other options, including doing nothing.

If my market validation consists of asking who likes my demo, that is puppy validation, a yes-no with no commitment. Of course, people will try me if the risk is low. I may even sign an agreement with a distributor to include my offer as a bundle of other offers, especially if the distributor does not need to pay upfront. Everyone likes a puppy until it poops. “Trying me out” does not provide valid data about why a user chooses me, in the same way as trying a free sample at the grocery store does not give meaningful data about who specifically will buy. I may get some aggregate data such as a general uptake in sales the day the free samples are available, but precious little info which helps understand why they chose me and if the choice is a committed one.

There is no generic metric that demonstrates that you have achieved problem-solution fit. However, a valuable signal is when you discover a metric that provides a unique insight into the user’s job-to-be-done.

Provender was a startup based on the idea that there is a lot of waste in bringing local-grown produce to the end consumer. Their initial product was an app where people could order direct from farmers. Of course, everyone loves fresh, local produce!

There was some initial traction, but nothing solid nor sustainable. Provender started to dig into who was ordering, and most importantly, why. They discovered a specific subset of users were chefs at small restaurants. Usually, they were forced to buy from brokers, the same Sysco trucks that go from restaurant to restaurant. Quality was mediocre and prices high. And most importantly, there was little leeway for small restaurants to distinguish themselves from each other based on product.

By digging deep, Provender found a laser-precise target user with a real job-to-be-done: chefs of small restaurants who wanted access to local, farm-fresh ingredients. Provender changed the app to help farmers list what was available today and what crops were soon coming. Chefs could choose what they wanted and signal their needs for future menus. Provender discovered valuable metrics that added value for both parties in doing so. The adoption of the revised beta in their test market was unanimous. They found the problem-solution fit. It also paid off for Provender because they quickly raised the seed round that enabled them to generate actual revenue.

The pressure to use metrics is enormous. But you need to choose the right ones at the right time. An increase in revenue during H1 is noise. During H2 (Capability), revenue increase starts providing helpful information because this is the stage to seek product-market and initial pricing fit. And revenue increase is a primary signal at H3 (Cashflow).

Too many founders see the initial revenue uptake as the signal that they are hyper-scaling. Metrics in the hundreds of percent are cool. However, any number divided by zero is infinity… and infinity does not exist.