Venture Hacks

Eric Ries and I recently sat down to talk about minimum viable products: the product with just the necessary features to get money and feedback from early adopters.

The minimum viable product (MVP) is often an ad on Google. Or a PowerPoint slide. Or a dialog box. Or a landing page. You can often build it in a day or a week.

I recorded the interview and synchronized it with some simple slides below. That’s my favorite way to consume the audio. You can also find a transcript and stand-alone audio below. Eric also highlighted some excerpts from the conversation on Lessons Learned and put together his own presentation on MVPs. Let me know what you think — I’m especially interested if you like the synchronized audio and slides.

In Part 2 of the interview, we discuss opening board meetings to the entire company.

Slides: What is the minimum viable product? (pdf)

Audio: What is the minimum viable product? (mp3)

What is the minimum viable product?

Nivi: First of all, this is Nivi from Venture Hacks, and I’m talking to Eric Ries from… where are you from?

Eric Ries: From the Lessons Learned blog.

Nivi: From the Lessons Learned blog, formerly from IMVU, formerly an advisor to Kleiner Perkins. We’re just going to have a discussion on a few topics of Eric’s and my choosing.

Nivi: First topic: What is the minimum viable product? Talk to me about minimum viable products.

Eric: OK, well let’s start with the question. Why do we build products in the first place?

In the end, we hope to be able to launch product to lots of customers and have them give us money so that we build a great business.

One approach to solving that problem would be, let’s build a product with the maximum number of features that will maximize our chance of success in the end. But the problem with that is you won’t get any feedback until you’ve already built all those different products.

All those different features, so you ship this product with a ton of features, and generally, by the time it’s done, it’s too late to make sure that you are on the right track.

The alternative would be, let’s do the release early, release often thing, and let’s get feedback as we go. The issue there is, if you just follow the release early, release often mantra, you find yourself running around in circles, because you ship code, you get some feedback from people, you do a focus group.

Some customers say, “Give me feature X,” “Give me feature Y,” now you’re kind of like, maybe sometimes you do what they want, maybe sometimes you’re going to do what you want, and then they get mad at you, and you’re chasing your own tail a little bit because you’re not operating against a clear, long-term vision of what you’re trying to accomplish.

The idea of minimum viable product is useful because you can basically say, look, our vision is to build a product that solves this core problem for customers, these kind of general feature areas, and we think that for the people who are early adopters for that kind of solution, they will be the most forgiving.

And they will fill in their minds the features that aren’t quite there if we give them the core, tent-pole features that point the direction of where we’re trying to go.

The minimum viable product is that product which has just those features and no more that allows you to ship a product that early adopters see and, at least some of whom resonate with, pay you money for, and start to gave you feedback on.

Nivi: So there is some set of customers out there, we believe, that just with these features alone, this product is useful to them.

Eric: Exactly right. And sometimes it’s useful to them because early adopters have the same kind of visioning power that entrepreneurs do, but because they can see what the end product is going to be.

Getting developers on the IMVU platform with a MVP

Eric: There are cases where you ship them a product that actually doesn’t work very well — at IMVU, the first version of our product worked pretty terribly, but, for example, it was all about user-generated content. We wanted to get developers on the platform.

The problem with developer platforms is this chicken and egg problem. Developers don’t want to develop unless there are customers who are there to buy their products, and customers don’t want to come on the platform unless developers are there selling them something useful.

What we did is we took early adopter developers and we told them a story about how IMVU was going to take over the world and be this really powerful product for mainstream customers and we made them believe it.

And we gave them an economic incentive that said, the earlier you get on board with the platform, the bigger your take is going to be for derivative products that get created down the road.

We shipped a product that basically had almost no customers — certainly no mainstream customers, and the developer tools weren’t that great — but, because we had told that story effectively and we really understood those early adopter developers, we got a ton of them on the platform developing.

Because they felt like they were in the middle of a gold rush, despite the fact that there was really no evidence to support that belief other than their own power of imagining what this thing was going to be down the road.

Luckily, we delivered on that vision, and so they actually were — a lot of them — pretty happy.

A non-MVP: The Kerry vs. Bush avatar

Nivi: I want some examples of these crazy minimum viable products. By which I mean, for example, and you can talk about any of these, the AdWords approach, the approach you talked about in terms of dialogue boxes and just popping those up, and just trying to sell a PowerPoint slide to an enterprise customer.

Eric: Let me start with an example of a time we didn’t know the minimum viable product, to illustrate my point.

It was 2004 — you have to remember 2004, Bush versus Kerry election. At IMVU, we had this idea — it happens to entrepreneurs all the time, you wake up in the shower and you’re like, “I’ve got an idea for a killer product.”

The idea for us — this was probably September/October 2004, debates are happening, politics is in the air — our idea was, we’re going to sell presidential debate avatar set that we would either dress up like Kerry or Bush, and you’ll be able to debate with your friends in this 3-D presidential debate product.

Now, put aside for a second whether that actually seems like a good idea to anybody else. We convinced ourselves it was a great idea, and we spent two full weeks racing to get this thing built, because it was time sensitive, the election is coming, the debates are happening, every day counts.

We thought it was a great idea for that whole two weeks. We built it out and we shipped it and we spent countless hours debating exactly what features had to be in it or not in it, and we were going to sell it for $1.99.

Our theory was, pricing won’t get in the way of anybody buying this thing. We want to make it cheap and easy and it will make lots of buzz and Wall Street Journal and New York Times are going to cover this thing; it’s going to be awesome.

I remember for us in those days, two weeks of development was a lot, because we were a pretty fast team. We did that, we shipped it — cut to the chase, nobody bought it. We sold exactly zero copies of the Kerry vs. Bush avatar.

We tried a bunch of different permutations and different variations of it and added features, and we changed the price, and eventually we gave it away for free, and even at free we couldn’t sell any copies.

Nivi: Well how many bought it for free?

Eric: None.

We literally couldn’t give this thing away. This thing was a dead weight money loser. There was actually nobody in AmErica interested in having a presidential debate avatar.

How to turn the Kerry vs. Bush avatar into a MVP

Nivi: How would you have approached this exact same product by taking the minimum viable product approach?

Eric: Well it’s interesting. We thought we were taking the minimum viable product approach because we had only spent two weeks on it. Right? Where we had made a very early prototype and put it out there.

But, if you think about it, going back to the definition of the minimum viable product, which is the minimum features that are required to learn what customers want, we had spent way too much time on it.

What we should have done, and what we did for a lot of features thereafter, is started with a landing page that promised people that product. Then we should have taken out the AdWords we were planning to take out, drive traffic to that landing page, and offer people to buy the experience that we are talking about.

What we would have found out if we were doing that experiment is 0% of people would have clicked through, which means it doesn’t matter what is on the second page.

The first page is so bad, not because it is badly designed, but because the features are wrong that you don’t need to go through the effort of building out the product. So we wished we had done that, and we did make that mistake really.

Nivi: How would you have marketed that minimum viable product to your existing customers?

Eric: When you already have customers you need to have some way to experiment with making them an offer. In a lot of cases the minimum viable product really is just that offer.

You can crisply articulate to customers what they’re going to get and how much they’re going to pay for it. You can learn a lot by just popping up a dialogue box that says, “Hey, would you like this new feature?” or showing them a banner ad for that feature.

For example, on IMVU we would have a system setup so that we could arbitrarily from the server select a small percentage of customers and make them an offer by inserting a dialogue box into their conversation, and basically it’s a simple “Yes” “No,” would you like this thing, “Yes” or “No?”

When they say “Yes,” we take them to a landing page where we try to sell it to them. Eventually if they really did want it, we would have to make up some excuse why we couldn’t give it to them like, “We’re experiencing technical difficulties right now.” “We’re not quite ready to give it to you. Give us your email address and we’ll email you when it’s ready.”

Again, you’ve got to remember that 99% of the time nobody wants it. Most offers that appear to an entrepreneur as a good idea are actually horrible, horrible ideas. By making the offer and having it be rejected by customers, we learn not to waste time building stuff that nobody wants.

Nivi: Right. Maybe the right definition of a minimum viable product, like you were saying, is, essentially a test to see whether people will actually want the product that you’re imagining in your head.

Rejecting false negatives: “But my customers don’t know what they want!” #

Eric: That’s right. The reason why this is an art and not a science is, I’ll have entrepreneurs come up to me and say, “But hold on, my customers don’t know what they want. If I ask them ‘Do you want this thing?’ They might say no when the answer is really ‘Yes.'”

Unfortunately, that’s an excuse that is used way too often, but there are situations where it’s true. The judgment call is; what really is the minimum set? In some cases like in entertainment products it might actually require you to build an early prototype, or a mockup, or even version one of the product with the minimum possible set of features that you think could go.

The nice thing about minimum feature set is you can always try intermediate points to ask yourself, “Am I at the minimum feature set yet? Am I at the minimum feature set yet?”

As long as you’re not afraid of the false negative, that is, if you don’t get discouraged because you’ve built your first paper prototype of it and shown it to people and nobody wanted it. That can’t mean that you give up because, “Oh, forget it, we’ll never make it.” You’ve got to say, “OK, well then let’s iterate some more.”

If you keep iterating at it, you keep making it a little bit more sophisticated, at a certain point after you’ve been through 10 iterations, that you still got no uptake whatsoever, and the feedback you’re getting from customers is still a yawn, you might say to yourself, “You know what? We’re not moving in the right direction. In fact, we’re past the point of minimum viable product. This just isn’t a viable product.”

Nivi: Right. Back to your point about entertainment, in Hollywood they start off in scripts. If the script looks good, then they will make a pilot. If the pilot looks good then they might order a few episodes from the first season. After the first season, then they make from there. They don’t build all three seasons, and then try to ship them.

Eric: Yeah, which means that some great shows never get made, because the early tests look negative and the people involved don’t have the courage or stamina to see it all the way through.

On the other hand, sometimes people have the courage and stamina to see through a really bad idea. That’s why the concept of learning is so important. This cannot be done on a spreadsheet. You have to keep training yourself on multiple iterations and multiple attempts to start to develop good instincts for it in your particular domain, your particular market space, what’s likely to work and not work.

Then you can do that as an entrepreneur, but even more powerful is if you can get your whole organization, everybody, training themselves constantly to do that kind of learning, so that good ideas will be passed around the whole organization.

Building products like packets get routed on the Internet

Nivi: Also minimum viable product ties into the concept of “build it before you sell it.” Are there any other out of sequence things that you guys did at IMVU or elsewhere that you found helpful? For example; you guys probably wrote tests before you coded to some degree as well, didn’t you?

Eric: Yeah, that’s right. I mean test driven development is the same principle.

Nivi: I’m wondering if there are some other things that you guys have done.

Eric: If you think about the full range of the product development life cycle; from specification to design, to implementation to testing, to maintenance to sales to deployment.

Nivi: Deployment and all that stuff.

Eric: I now think about that rather than as a linear sequence, I actually think about it as a big network. The question is; just for any given feature, in what path should you route it through that network? Different features should be routed differently, just like on the Internet we route packets differently as necessary.

Nivi: I’m wondering if you have some specific examples of paths you’ve taken.

Eric: Yeah, we just talked about writing tests first. We’ve talked about selling first. We’ve talked about deploying first.

Deploy first, code later

Nivi: When do you deploy a product? When have you deployed something before you built it?

Eric: All of these forms have offers of products. For example in a lot of Internet products, how does the customer know that the feature exists? Yes, there might be 100 pages of fancy stuff, but how do they even know that stuff exists?

Generally, it’s because there’s a link in the header. There’s one link on one key page that notifies them that they can go do this other thing or at a key moment they receive an email, even though the feature might be wide and complicated.

Generally, access to the feature is controlled through a simple choke point. We would often add those choke points in a split test just to see first of all, did anybody click on them? So we could look at the click-through rate of people that now believe there is this feature.

We would also see an interesting phenomenon, sometimes the presence of a feature, even if nobody clicks on it, still impacts their behavior in other ways.

For example — I can’t remember the data, we saw an experiment where, this isn’t exactly right but something to this effect, we added a link to the header notifying people that there was a VIP Club for special people only to get access to on IMVU.

Now, it’s not the same as what we have now. IMVU does have a VIP club today that is not anything related to this. But, in those days, the idea was, we were trying to test whether people wanted to have some kind of premium experience.

What happened was few people actually clicked on that link to go find out about the VIP club, but in the experimental cohort for people who were shown that header, their average spending was higher.

By constantly reminding them that there was such a think as a premium experience, we primed them to want to do more spending on IMVU. It was a very unexpected result.

And it’s a good example of why you always do full cohort-based split tests. Always test for the macro effect, don’t look at the micro effect. That’s a case where deploying a feature before we’ve built it actually can give you an insight that maybe you don’t need to build a feature after all.

Nivi: Interesting. Yeah. That reminds me of an example that one of our friends told us about where they couldn’t get people to sign up for a free product, so they put a paid version of the product right beside it, and I don’t think anybody signed up for the paid version…

Eric: It made the free version look a lot more…

Nivi: Yeah. It made the free product look legitimate. And they started to actually convert on that basis.

Design first, code later

Eric: There are also cases where you want to design a product last, rather than design it first.

We would often ship things in a schematic form with horrible design to see if we’d gotten the information flow and information architecture right, and really good interaction designers, if they’re being honest, will tell you that that’s always the sequence.

You always re-factor your design out of specific use cases and out of specific uses, rather than starting with the broad vision of what it’s like.

Nivi: Basically the trails where the people walk, you pave those?

Eric: Exactly right. And you could do that — I remember one time we used to use paper prototypes where you have designers sit down with a focus group of people and show them a fake screen shot that they came up with in Photoshop.

And then the customer says, “I want to click that button,” and you go grab the piece of paper that represents what happens when you click that button, you show them another prototype, and we used to do them that way.

We actually built a paper prototyping website into our website where, for certain customers, the designer could come in and actually replace sections of the website with full-on mockups from Photoshop that were sort of clickable and sort of functional.

Then we would bring a customer in and have them register an account; we would specify that that account should see the paper prototype, and then we would go through with them, do the focus group.

But the customer doesn’t know they’re seeing something that’s a mockup. They think they’re using the product.

And, yeah, the product is a little weird, because sometimes they click things and nothing happens, and sometimes things won’t look a little right, but customers think that about your product anyways, so it’s not particularly abnormal for them to see a product that looks completely incoherent to them. That’s the state-of-the-art for most customers and those products.

That was another case where we would — and we would, occasionally, come up with some prototype that that functioned well enough that we would ship it to real customers while we’re working on the full on, blown, beautiful version of it.

Sometimes customers would see a schematic version from which we would gather more information about whether we’re on the right track, and put things in the right place.

Where did you get the money to experiment?

Nivi: What allowed you guys to do all of this experimentation? Specifically from a financial point of view? Not about the mentality behind it and so on. Time is short and money is running out, or were you basically break-even this whole time, or how did you work that?

Eric: No, we believed in a concept we called “bridge to profitability, ” which is, whenever we raise money, we always raise money on the basis of a plan that we honestly believe, would get us to profitability on just that round.

If you notice, that doesn’t mean that we actually got to profitability on that round. In fact, we kept raising more rounds because we kept believing that we had the opportunity to pursue a bigger opportunity by raising money, but we never had to raise money. We always felt like we could get there on our own.

But that meant that we were constantly a dollar short, because plans to get to profitability on a round generally involve you going negative and then catching up before the money runs out, so we were not eager to learn, eager to experiment.

We did not love metrics. We were very scrappy and very afraid of running out of money, so we only ever adopted practices that we thought would have a high ROI.

The issue is that these practices do have a high ROI, and we got to them because we kept following the traditional model and failing and feeling like, God, we are burning money like there is no tomorrow. We keep building features that nobody wants. We literally don’t have the time and we can’t afford to keep doing the same old thing over and over again.

I don’t know if we were just lucky that we had that mindset, but, for whatever reason, even though at some level we always believed the next feature we shipped was going to save the day and have a step change in our profitability and take us to the moon, because it’s just the best feature since sliced bread.

Because we were so desperate to actually make money, we — I’m trying to think of how to put this — we became obsessively committed to actually being right rather than just believing in the next thing to save us.

If you look at the traditional hockey stick shaped curve, it’s flat for an awful long time. We had been in a company previously where we were always promised, “Yeah, it’s flat now, but we’re on the verge of the hockey stick.”

The problem with the hockey stick plan, when that’s your plan, is you just work for a year and then the 13th month you’re going to go to the moon. For the 12 months, you don’t know if you’re making progress or not, because you’re following the plan.

Everything is flat like it’s supposed to be, because when the final feature comes into place, you’re going to go to the moon.

We just didn’t believe that was going to happen. So we forced ourselves to actually test each feature to see if it was going to be successful.

Then we just stopped being able to drink our own Kool-Aid, because we were just wrong so often, it became harder and harder and harder for us to convince ourselves that it was a good financial investment to just randomly try the next new thing versus actually trying to say, “Hey, is there a way that we can increase the probability of having a successful feature.”

Nivi: Yeah, and I think, there are probably two other things here, right? You charge from day one, so basically every experiment you ran was an experiment to make more money, right?

Eric: That’s right.

A preview of part two

Nivi: Why would you not run experiments if you’re charging from day one. And then, the second thing is — this gets us to the second topic, which is: You guys weren’t doing any of this really in public because you had not launched the product, right?

Eric: Amen.

Nivi: Nobody knew who you were. But people, at the same time were using the product, so how did you do that?