The startup was one of those genuine “rocket ships.” That didn’t come without its costs, like the engineering organization being long overdue for a reshaping to support growth and business needs. But the CTO, a first-timer, was spinning his wheels for months, torn.

He felt like he didn’t have the solution that was “just right” and so got paralyzed. He was afraid of promoting a solution that would turn out to be wrong. Yet together, we made that change happen in a matter of weeks. The difference wasn’t my brilliance, as much as I’d like to claim it. It was learning how to experiment.

Not Having The Answer

Your fear of committing to the wrong approach is very understandable. Yet most of the time the answer is not to keep waiting till you find that perfect solution. Instead, you should embrace the uncertainty, realize that it’s here to stay, and learn how to move forward anyway.

This starts with allowing yourself not to always have the answer. No one really expects that except you. I mean, you don’t expect your CEO to have all the answers for what the business is supposed to do, exactly which features the market will rave about, etc. Why can’t you cut yourself that same slack?

When we don’t have an answer, but we have an educated guess, in most cases that’s enough to start moving forward. We define it as an (articulated) experiment and get on with it.

Advantages of Experimentation

Why do I love experiments so much that I keep telling my clients they’re not utilizing the experiment hack enough? Because they have so many benefits. For example:

• They protect egos: When you plainly state that something is an experiment, you’re allowing for it to turn out to be a mistake without putting all your eggs in that basket. You’re getting some protection. For some people, this can make all the difference between being frozen and taking action.
• They protect the team: My client’s much-delayed reorg we started with was due to his fear of the team living through an approach that wasn’t ideal and then losing all trust in leadership. When we let them know right from the start that this is an experiment and thus will likely require iterating and even could fail, we also prime their thinking.
• They remove objections: Every large enough engineering organization will have a couple of senior engineers who like to poke holes in ideas and initiatives. But when you don’t claim the solution is perfect, it turns objections into ideas for improvement. Also, when things are viewed as not permanent, many people are more open to seeing how things work out.
• They encourage speaking up: The opposite of the previous point, about those who actually don’t talk. Some people find it hard to voice their concerns about an idea formulated by “the boss.” When you yourself describe it as an experiment and say that it will require tuning based on feedback, you’re making it much easier for them to speak up.
• They induce learning and improvement: If you were just aiming to find a really good approach and go with it, you’d risk getting stuck in local maxima. With experimentation, you’re actively iterating, making it much more likely that you’ll end up with a better final approach.
  

How to Run an Experiment

Now that you’re clear on the many benefits of experimentation, let’s get you started. Whatever your first issue is, come up with a good enough approach and start executing:

1. Define it: State the problem that the experiment is intended to solve (or improve). It should be clear to everyone what the importance of the problem is that merits this intervention.
2. Explain your reasoning: What is the hypothesis behind choosing that specific approach? How will success look?
3. Create metrics: How will you monitor progress as the experiment is ongoing? What are your leading and lagging indicators? Communicating these is greatly helpful in getting people to speak up, because they can use your own metrics to explain their issues, making it easier.
4. Time-box it: It’s not an experiment if it’s indefinite. How long are you giving this before you decide what to do? It can be a week, a quarter, or even more, depending on the issue you’re tackling. But be clear about it, and ensure that you have the right reminders set up in your calendar to ensure that the experiment will not just be forgotten.
5. Set up feedback platforms: For a short experiment of just a week or two, you might find it sufficient to schedule a retrospective at the end of it to decide about the next iteration. For longer ones, it’s useful to have recurring syncs (or asynchronous feedback collections, based on metrics already discussed but not limited to them).
6. Wrap up the iteration: At the end of the predetermined period, document what you’ve learned, what worked and what didn’t. Communicate what the next step is (leave it as is, revert the whole thing, experiment with a tweak or an entirely different approach, etc.). This public closure is vital to make people believe your experiments are really that.
7. Rinse and repeat: Keep iterating on that specific issue as long as it makes sense. All good? Move on to the next thing!
   

As my client saw, rapid change is possible and often much less scary once we adjust our mindset. Stop aiming at perfection and use the same iteration that moved us away from waterfall and to more agile ways decades ago. It’s not limited just to code. And to experiment more effectively and improve your speed of change, don’t hesitate to reach out to me.