Optimization Mistakes that Kill Startups

I once believed optimization was the secret weapon that could make almost any startup successful. It was certainly a critical part of reaching millions of users in each of my first five startup marketing roles. At a couple of startups we saw a tripling of conversion rates from a single experiment. When we tripled conversion rates, we tripled the effectiveness of every future marketing dollar.

I first became a fan of funnel optimization at one of my early startups where we had hit a wall trying to develop scalable customer acquisition channels.  We decided to temporarily stop trying to find new customer acquisition channels and focus instead on improving conversion rates.  A few months later we resumed channel building and were able to scale the same previously tested channels to support 100X the marketing spend with the same target ROI per dollar spent.  Beyond the clear benefit of enabling scalable marketing campaigns, the improved user experience also resulted in a multifold increase in free organic growth.  User growth immediately hockey-sticked and years later still  hasn’t diminished.  All the while, the company maintained cashflow positive results.

These benefits probably have you chomping at the bit to start your own optimization program. But be careful, optimization can easily kill a startup when not done right (or at the right time).

Here are the three most common optimization mistakes startups make:

1) Premature optimization – Optimization is about improving the path that users take to reach a certain destination within your website. For most sites it’s ultimately about getting people to experience and buy your product. While this seems like an important goal from the beginning, it’s not. If the value of your core product is weak, doubling the percentage of users that get there won’t help much. And it will actually hurt you because every unit of effort put into optimization is one less unit that you can put into improving your core product. Products that don’t become a “must have” almost always fail.

My recommendation for startups is not to begin optimizing until at least 40% of your randomly surveyed users say they would be “very disappointed” without your product. That doesn’t mean you shouldn’t try to have a great first user experience, rather it means you shouldn’t start iterating flows until the core product meets this threshold.  The only exception to this is if your value proposition will increase because of a network effect (like eBay). I’ll try to write a post on this scenario soon.

2) Not being deliberate –To execute full funnel optimization you test multiple changes at every step in the acquisition process. Since every change is also an opportunity to screw things up it’s extremely important to measure the actual results of a change. Unfortunately traditional analytics programs aren’t helpful here since they don’t track specific user cohorts moving through the funnel (AKA groups of users). In the early startups I worked with we spent months building systems internally to track conversions at the user level. Fortunately “off the shelf” systems are now cropping up that make user level funnel tracking much easier (I’ve been advising KISSmetrics on such a system for over a year and I’m now using it in a couple projects). With the right system you can track your “measures of success” and roll back any changes that havea negative effect on these metrics.

This presents a new problem. Anyone with a basic understanding of statistics will realize that optimization is a numbers game. If you test enough things you will definitely find something that improves your key measures. That’s the theory, but the reality is that you’ll never get past the first few tests if the early ones don’t yield improvements. People quickly lose faith in the process. Therefore it is essential to vet every test idea before asking the development team implement it. Prioritize test ideas so that the easiest and/or most likely to improve results are implemented first.

3) Killing the love – One thing that is rarely measured in an optimization project is a reduction in the core value perceived by your most passionate users. Your ability to deliver an experience that creates passionate users is your most important asset as a business and must be protected. It can be improved, but it must be done very carefully. The first step in protecting it is to understand it. I never attempt an optimization project without first doing a project that helps me understand the use cases of the most passionate users. After this initial project, which I combine with messaging optimization, I am in a much better position to safely optimize the full conversion funnel.

Effective optimization requires the right tools, qualitative research/understanding and a systematic approach to testing. When executed properly it can easily result in 2X – 10X improvements in conversion rates. No business will come close to its potential without a concerted optimization effort, but be careful to avoid the mistakes listed above.