Dropping Vanity Metrics

🎶 You’re so vain, you probably think these metrics are about you 🎶. Having just wrapped up a series about profitable R&D organizations, I realized there’s one added aspect that we need to cover, and that’s aligning your own personal sense of progress, achievement, and impact with actual business value. To do that, let’s quickly go over vanity metric pitfalls I see at startups regularly.

What You Measure Gets Managed

You know the saying, we have to measure something to improve it. Yeah, that makes sense. But let’s turn that around. You might be measuring the wrong things, meaning you’re focusing on things that are not helpful or managing them so well it is to your detriment. Therefore, try to keep yourself honest and aware of what you’re optimizing for.

Here are some examples of bad metrics:

  • Number of pull-requests: Let’s start with something basic. Some “engineering velocity” tools make teams focus on things that are wholly useless. I saw teams bragging about having improved their number of PRs each sprint. Did it necessarily amount to better results? No, just extra overhead as people became dogmatic about splitting everything into lots of different PRs, increasing overhead needlessly. Don’t get me started about those who don’t need to work to get to higher numbers because their uselessly complex microservices architecture requires a dozen repos to be touched for any change.
  • Headcount: Perhaps not as popular as it used to be during the 2021 high times, it’s still something I hear mentioned in many executives’ measures of success. I’d always rather a small but impactful team over a larger team that’s not getting results done. See more about profitable organizational structure here.
  • People never quit: Or, perhaps “we never had to fire anyone.” Similar sayings might sound nice, but in fact, what they’re making is encouraging leaders to contort the organization to fit people long after it no longer makes sense. Your founding team might not be the best to lead an IPO-ed startup. Never firing anyone means you’re letting people stay even when they’re not the right ones for the job or are so risk-averse in hiring that you’re never staffing positions.
  • Hitting 100% of goals: Don’t get me started about this. If it happens occasionally, that could be a great win. But if this becomes your expected mode of operation? It just means there are too many layers of buffers padding your plans and no innovation, creativity, or stretching done. Don’t lower the bar so far that it’s too easy to reach.
  • Seniority of staff: Oh, I like this one. Are you one of those touting a team full of seniors? The truth is that years of experience are not always a good yardstick for productivity. Moreover, doing so means you’re losing out on the ability to onboard great rising talents and help shape them to fit your company. It’s sort of like the sports team manager who is outbidding everyone else to draft top people.
  • Needlessly high bars: For example, you might be demanding so many “nines” of availability from your system because that’s what “you’re supposed to do.” However, when considering it seriously, you’re just working on a B2B SaaS that doesn’t supply real-time results and could very easily have more lax demands that would enable your team to focus on things that actually matter to users. Too much of a good thing, you know?

Take Control

Instead of following these metrics that are fun to boast about on LinkedIn, take the time to consider what metrics would actually move the needle in your specific case. What’s required of your team to help push things forward and enable business success?

I think there’s something very freeing and energizing about pushing aside “common metrics” and doing more contrarian and special things. These can really rally a team around something and help create something unique. If you intend to build a team worth leading, make sure to use your own measures of success and not anyone else’s.