Thanks to KISSmetrics for supporting our interview with Sean Ellis. If you want an intro to KISSmetrics, send me an email. I’ll put you in touch if there’s a fit. Thanks. – Nivi

Hiten Shah from KISSmetrics recently sat down with me to discuss how to optimize funnels with their upcoming analytics tool, KISSmetrics.

You may know Hiten from his Crazy Egg days and survey.io, which Hiten and I discussed in How to measure product/market fit with survey.io.

SlideShare: How to optimize your web apps with KISSmetrics
Audio: Interview with chapters (for iPod, iPhone, iTunes)
Audio: Interview without chapters (MP3, play anywhere)
Transcript: See below

Prerequisites

You’ll get more out of this interview if you also read:

1. Our interview with Sean Ellis parts 1 and 2.
2. Our previous interview with Hiten: How to measure product/market fit with survey.io.

Outline

Here’s an outline and transcript of the interview. The interview and transcript are about 23 minutes long and we’ve highlighted some of the juicier bits for you.

1. After fit, prepare for growth
2. KISSmetrics helps you optimize your funnel
3. KISSmetrics helps with all optimization steps
4. First user experience isn’t necessarily the paid user experience
5. KISSmetrics is most valuable after fit
6. How Cloudfire uses KISSmetrics during fit
7. Startups aren’t a science — but we’re getting closer
8. How other people use KISSmetrics
9. KISSmetrics is on it’s third iteration
10. Why KISSmetrics’ hasn’t launched
11. KISSmetrics tracks actions on a per-user basis
12. KISSmetrics lets you calculate customer LTV
13. Get more startup advice

Transcript

Music: Squarepusher

Trent Wolodko: Hey there everybody, my name’s Trent and I’m new to Venture Hacks. I’m going to be helping with producing content for the site so that the guys can concentrate on bringing you good advice faster and more furiously than ever. Today Nivi speaks once again with Hiten Shah from KISSmetrics.com. He spoke with Hiten recently about survey.io and today  he’ll be talking to him about optimizing your web apps using the tools over at KISSmetrics.  Right off the bat, these guys make a lot of references to Nivi’s interview recently with Sean Ellis about bringing your product to market as well as his former interview with Hiten regarding survey.io. So, you may  want to check out those interviews as well on the site. The guys have a lot of ground to cover as you can see, so without further adieu, I’ll throw it over to them…

Babak Nivi: So the second part of the interview with Sean is what activities you do to prepare for growth. Right?

Hiten Shah: Yeah.

After fit, prepare for growth

Nivi: One of the insights that I got from the interview was that you do a whole bunch of stuff to get to fit. That’s step one. Then step two is that you do a whole bunch of stuff to get ready to prepare for growth.

Hiten: Yeah.

Nivi: And Survey.io is great for the product/market fit stage, where you’re trying to get the fit. And you guys also have a product called KISSmetrics, which helps you prepare for growth, or to put it another way, to optimize.

Hiten: Yup, and that’s our main product.

Nivi: So, what does it do?

KISSmetrics helps you optimize your funnel

Hiten: First, it’s an analytics product, pure and simple, but the difference is that it’s meant to do one thing really well, and the thing it does really well is help you optimize a funnel – help you understand what your funnel looks like for whatever type of product you have.

Nivi: So, Sean has a few steps in the optimization stage, right?

Hiten: Absolutely.

KISSmetrics helps with all optimization steps

Nivi: Economics: put into place a business model and optimize it. For example, change your price or change the way that you charge, completely, whether it’s by the year or by the pound or however. So that’s the economic step. That requires optimization.

There is the positioning step – how you describe the product – which also requires optimization.

And then there is the step of essentially moving people through the funnel and getting them to the finish line. That requires optimization.

Hiten: Yup!

Nivi: How do I use KISSmetrics in any of those?

Hiten: You can use them in all of them, because when you have economics in place it means that you’re charging customers and that you need to understand things like how different price points affect your revenue. In essence, it’s a funnel. You’re still funneling people through a sign-up process, and you might be A/B testing two different things.

So we facilitate some level of A/B testing today, because you can pass in the data and visualize it, but in the future we’re going to start adding more components around A/B testing and optimization. Right now it’s a hard focus on helping people understand their funnel, but then that goes into both of those areas we were talking about.

So, in the economic stage it’s just about when you’re testing different pricing plans and things like that, and understanding if there are any differences, also, in different channels of customers depending on where they came from, and if your revenue is different for those based on the price points.

Then on the promise side you’re probably doing a lot of things around testing landing pages and things like that, so you’d build different funnels for different landing pages and different cohorts of users, and we completely facilitate that, as well.

Nivi: So you let me – and maybe you can just send me a couple screen shots that I can put in – compare two cohorts, how many people get to the finish line based on their channel, based on the positioning that they saw on the landing page, based on the final price or business model that they saw that we offered them, and also based on what Sean calls that “first experience.”

Hiten: Yeah, first user experience.

First user experience isn’t necessarily the paid user experience

Nivi: The first user experience, which is just kind of the funneling. I mean, it’s everything.

Hiten: Yeah, but it’s primarily how to optimize your on-boarding process of first-time customers so you can make that more efficient – efficient meaning highest conversion rates – and so that you can spend on marketing and you understand what the value is of each user.

Nivi: What does that mean to you, other than a conglomeration of the landing page and all the steps between that and putting the credit card in or signing up?

Hiten: In every product there are a number of steps people take to get gratified with the product. Gratification, usually in freemium products, happens during the free period, most likely, that instant gratification, and then they have another reason to go pay. So it’s all about understanding how people are moving through those stages of their usage of your product. So the idea is to track all the different events of gratification and optimize against that. And there are always steps in between before they’re gratified.

For the first gratification, they have to sign up. They probably have to fill out a form. They probably have to install something, or depending on what type of product it is there are a number of things they need to do before they feel satisfied that this is a must-have product. And once they hit that point, then you’re going to have an ask of asking them to upgrade, and making sure they hit a point where…. It’s really about you trying to understand what point they hit before they’re willing to give you money, and understanding the different things you can do around that to encourage them to give you money.

This all has to do with, if you have free plans, you’ve optimized your first user experience. Now you really need to understand what the prompts are and where you put them to get them to pay you money. So it’s all about prompt optimization.

An example is if you have an analytics product and you have a free plan, but you want to charge users, you probably need to put prompts in different places, as customers do different things with the product, to get them to pay, basically.

Nivi: In order for KISSmetrics to be useful to me, should I be done with the product/market fit stage. In other words, should I have 40% must-have users, or can I use KISSmetrics earlier, effectively?

KISSmetrics is most valuable after fit

Hiten: The short answer is that I think KISSmetrics is most valuable after product/market fit. But, this is one of those things where creating a startup and the steps to creating a startup aren’t defined yet. People look at them in all different ways.

I think both Sean Ellis and Eric Ries are making a lot of noise and movement, and there are a few other people like Dave McClure that are doing all of this, to get people to think of it more as a methodology, and there are different stages to a company and different things you should do at each stage. So if you follow one of those methodologies or one of those mindsets or theories, what you really want to do is wait until you have product/market fit before you optimize first user experience.

My own example is that we’re not optimizing our first user experience for KISSmetrics right now. We’re getting users in, but we’re not optimizing it because, really, that’s not the important part right now. The important part for us is getting enough users and building and iterating on the product to make it so that it’s something they can’t live without. And that’s not a quantitative thing; that’s not something where if I can measure how they go through a funnel, it’s going to necessarily help them. It’s more about things I need to add in the product, and it’s a little more of a fuzzy goal. Honestly, hitting product/market fit, in my opinion, is a pretty fuzzy goal.

Nivi: Tell me how Ash, who has written about this…What’s the name of his company? CloudFire? Is that right?

Hiten: Yeah.

Nivi: How is he using KISSmetrics actually in the product/market fit stage? Is that right?

How Cloudfire uses KISSmetrics during fit

Hiten: Yeah. What he’s actually doing is combining a few ideas. Dave McClure has something called AARRR!! that’s a framework that he came up with years ago. He’s also one of our advisors, for a good reason. And he spread a lot of the religion of metrics, so a lot of people say he’s the church of metrics.

But basically, what it does is break down your business metrics for a startup into 5 different things.
Acquisition, which is metrics around how you acquire customers, what channels are profitable or what channels are bringing in customers and how you’re bringing them in. So it’s more like market.

Nivi: So you define acquisition.

Hiten: Yeah.

Nivi: One definition might be: they signed up.

Hiten: Yes, exactly. They signed up, and where they signed up from.

What Ash has done is taken acquisition and made a funnel for it, and he calls it the acquisition funnel in KISSmetrics.

Then the next one is Activation, which is how to get them to have their first gratifying experience. They’ve activated with the product. They’ve used it. They’ve experienced it.

Nivi: And in his case it’s something like, they downloaded my software and uploaded some photos with it. Something like that.

Hiten: Exactly! Exactly. It’s like they activate it, which means they downloaded it and uploaded a photo. They had the first experience of using it.

And then the next one is Retention, which is that they came back or they used it multiple times or some factor of that. It depends on what your startup is. So for him it may be that they used it once, but then they used it a second time and that would be retention.

Nivi: In thirty days, say, they use it again.

Hiten: Yeah. So he’s got another funnel around that.

And then the next one is Referral. That one is more like, are they referring other users? And that’s a whole thing in itself; it’s ‘forward to a friend’ and ‘contact importer’ kind of stuff. There are all kinds of things around referral.

Nivi: Tweet about it.

Hiten: Yeah. That also gets into virality and all of that kind of stuff.

And then the last one is Revenue. So he’s got some revenue funnels around all of the prompts. I don’t know if this is exactly what he’s doing, but this is what I would imagine he would be doing. I don’t look into his account; I just talk to him and read his blog posts.

Nivi: That’s good.

Hiten: He’s probably got a funnel that’s focused around the prompts that he’s using to get people to pay. So he’s actually using it [as an] analytics tool, in general. People end up using it whatever way they want, and that’s one of our challenges, which is that we really need to figure out what type of use-cases we want to support.

Nivi: Question number 6 or 7.

[laughter]

Hiten: Yeah. You got it. So we’re not even there yet of giving out that survey, but we will be soon enough, hopefully.

Nivi: I think it will be interesting to see how his activation and retention numbers tie into the must-have segment.

Hiten: Agreed. Yeah.

Nivi: Probably mostly on the retention, if I’m thinking about it right now correctly, because for the person to even be a user that you would survey with Survey.io, you’d probably want them to already be activated.

Hiten: You might even want them to be a retained user, and hit the retention step.

Nivi: Yeah, you’d want them to be retained. Right.

Startups aren’t a science — but we’re getting closer

Hiten: So I think he’s actually using it in a very interesting way that we’re probably going to study a little bit deeper, because Dave’s an advisor and we buy into his theories. Actually, our first iterations of this product, which not many people saw, each of those was a view in our system, so we’re actually very intrigued by what he’s doing with our product. But again, I don’t think startups are a science yet, and I think we’re getting closer and closer to it, so it just really depends on what people subscribe to. All we can provide is a tool at the moment, and then hone in on these different use cases of it.

Nivi: Right. One thing that Sean actually mentioned in the interview is that he doesn’t see a lot of correlation between the recommendation figure and “success of the startup.”

Hiten: Right.

Nivi: Whereas he sees a fair amount of correlation between the success and the must-have figure. So that’s one thing to keep in mind for Ash, perhaps.

Hiten: Sure.

Nivi: And then, too, just in your case, like you were saying, with KISSmetrics, the way you’re bringing it to market is very feature oriented right now, and then you’ll figure out the positioning based on qualitative conversations with customers.

Hiten: You’ve got it. Right now it’s more about product and how to tweak the product. I hate to call it features because for us, actually in the market we’re in, every analytics product has hundreds of features. We actually want to provide the features that matter, so that’s kind of the difference.

It’s basically iterating on the product and making it so people like Ash love it. There are probably a number of things… He might already love it. He might be a very unique case where he’s been looking for something like this. But a lot of other people, I get on calls with these people or talk to them, and….

Nivi: So how do other people use it?

How other people use KISSmetrics

Hiten: How they’re using it is that there are several Facebook applications that are using it. They’re using it to optimize their own funnels and just optimize the conversion rates between each step, and things like that – normally what you would do with funnel optimization.

And then other folks, like SaaS businesses that are using it, are buying traffic from AdWords and they’re passing in the dollar value for their customers at one of the steps, and they’re just basically optimizing the funnel, but based on revenue because that’s what their main goal is and they’ve already got their revenue set up.

Nivi: So, visualize the funnel and visualize cohorts, and compare them.

Hiten: Yeah. And some of this stuff is still things we need to do better in our own product. We’re very early. We released this version on November 1st, so we’re not even a month-and-a-half into it.

Nivi: You guys have basically realigned the company around this product, is that right?

Hiten: We’ve realigned the company around this product through doing our own customer development this whole time – “this whole time” meaning this is our third iteration of the product and this is the one that we’re being a lot more public about.

Nivi: What were the first two?

Hiten: With our first one we had assumptions around doing metrics for Facebook applications and social network applications. And what we realized there was that we had a very constrained view, and it was based on Dave McClure’s AARRR!! stuff that we just talked about. What we realized about that was that people started asking all kinds of questions that our interface could not answer for them.

So we took what we learned there and we built a second product, which was very super-flexible event tracking, and we had no funnel analytics or anything like that, so this kind of stuff would help you with tracking engagement of users. If you were a game it would help you track game plays by gender and all of those kinds of things. And if you were a SaaS app, it was more about if Ash needed to measure what types of photos people were uploading or what kinds of things they were storing on his system, we would provide metrics around that kind of stuff.

Nivi: And what was that called?

Hiten: It was KISSmetrics. All of these are KISSmetrics, we just didn’t publicly release them.

Nivi: OK, got it!

Hiten: And so we used these other two products to kind of learn about the market and figure out what our target should be. And it was super-generic products, especially the second one. The first one was very targeted towards a market, then we kind of ran away from that and said, let’s get Facebook apps, but let’s also get SaaS applications; let’s get blogs using our product. And that was a very highly flexible product.

In some ways it would have been really fun to market that product and deal with it, but we were just heading down a path, in terms of sales and marketing and things like that, that we didn’t want to. As a business our DNA was more around building a really simple solution.

We have a previous analytics tool that the people in our company built called Crazy Egg, and it’s a simple tool, and we wanted to go back to that thinking and say, analytics is something where people log into analytics systems, and a lot of the time if they’re not expert users they’re very confused about what they’re looking at, and then they end up just using it as a trending tool just to see general trends.

And the kinds of features that we felt were most important were conversion goals, conversion rates and funnels. So with all this learning from our previous products we basically decided that we should hone in on that proposition, and instead of tacking funnels onto an analytics product, like the other one, we would just make that the product. And it was a big bet. I’m actually super excited about it and very happy that we went in this direction because I think we can really make an impact on the market and produce something really useful for customers.

Nivi: The feature is the product, here.

Hiten: The feature is the product, and that’s a trend. I mean, Twitter is just a feature, right?

Nivi: Right.

Why KISSmetrics’ hasn’t launched

Hiten: A lot of people have said, “Oh, you guys are creating a lot of buzz. We know your brand, but what the hell is your product, and when are you going to launch?” And for us it’s not about that. It’s not about launch; it’s not about things like that. It’s about building a product that people love and iterating until you get that.

Nivi: Yeah, and this is a good example of why you probably don’t want to launch until the business is working.

Hiten: Yeah, exactly. We don’t even have our economics down, so we’re not charging anyone right now, and everyone can come in and use the product. At some point we want them to help us figure out what the value is and what we can charge for it.

The way most analytics packages charge, people are penalized for being successful, and we want to see if we can change that a little bit. I don’t know what it means yet. I don’t know what it means for our pricing model, but I’d love to make it so that if you get more traffic we don’t necessarily charge you more. It sounds really fucked up and weird, but yeah. Excuse my language, but we want a better pricing model if we can find it, that’s more aligned with our customers. So there might be more features that they pay for, and it might be access to data or something that’s not exactly like: you get more paid views or you get more growth and we charge you more money, and it scales up like that. Even for us, that’s not necessarily the most scalable business model.

Nivi: Sean talked about using action tags with AdWords, in the interview, to track users on a per-user basis. We talked about Google Analytics, which doesn’t really track users on a per-user basis. What does KISSmetrics do differently here?

KISSmetrics tracks actions on a per-user basis

Hiten: The things we’re trying to solve are if someone comes to the site multiple times and then they sign up, let’s say on the third or fourth time, we’re allowing you to identify that user and we’re trying to attach all of those times that they came, previously, to that customer. What that means is that we’ll understand the first touch point they had with you and where it came from, and we’ll be able to attach that for the history of that user using your product. And that’s a much different view than most analytics have, which is either page-view tracking or event tracking. This is actually, honestly, people tracking, which means that we’re trying to actually track people.

Eric Ries has a great quote on this which is, “Metrics are people, too,” and we firmly believe that. If we can track things on an individual-user basis without getting into a lot of privacy concerns, because you’re already doing this in your data base, so you just have to have the proper terms-of-use for it, we can provide a much more valuable analysis on your user base. And as long as we’re not doing anything in aggregate there’s not any real privacy implication, while with Google Analytics I feel like it’s very hard for them to do that because they have a huge business that has nothing to do with analytics, in a lot of ways.

Nivi: Right. Does tracking people on a per-user basis let you calculate LTV, and to put it another way, cost of customer acquisition, and does Google Analytics not let you do that?

KISSmetrics lets you calculate customer LTV

Hiten: I’m not going to say that Google Analytics does not let you do that. We definitely think there’s a lot of value in the product, and we actually use Google Analytics in a lot of our products, and they do different things than we do. But legally, you cannot pass personally identifiable information into Google Analytics. So, if you’ve got 100 conversions in Google Analytics, you might have 110 in your data base simply because you’re not able to identify each individual user and they might lose some of the data.

And they’ve got, my guess is, 100 million sites using the product. That’s a high load. We’re probably never going to have 100 million sites using our product. There are a lot of issues that come around dealing with that many customers with a free product.

Nivi: Yeah. What I’m just trying to understand is what the requirements are to calculate cost of customer acquisition. Have you guys thought about that at all?

Hiten:  Yeah, absolutely. We want to make it as simple as you passing a dollar value every time an event occurs where someone’s paying. And then in our interface, it’s not like magic, but it’s magic. We can show you revenue by referrer, revenue by channel, revenue by plan type, and all of those kinds of things that typically you would have to do a lot of data base queries in your own data base, your user data base, to get all this data out.

And honestly, this is some of the pains we had with our previous products. So, for the first 6 months, we never tagged referrer for each customer that came in, so we couldn’t boil it down to what referrers were creating the most revenue for us. Once we started doing that it was enlightening. So KISSmetrics is actually based on a lot of the pains that we had with our previous SaaS product.

Nivi: So the dollar value that you pass to KISSmetrics will let you calculate the LTV, to some degree, of the customer, right?

Hiten: Yeah, that’s the goal.

Nivi: But not on the cost side. It doesn’t tell you how much it cost you to get the guy.

Hiten: Not today.

Nivi: Right.

Hiten: That’s a really good question, and that’s something that we do want to answer in the future. Obviously it depends on what type of customers we get and how they want to pass in the data. Is it automatic? Is it manual? Things like that. But it’s a problem that we’re thinking about a lot because then you can have a full picture in one dashboard. It’s all about that one report to rule them all, which doesn’t exist in analytics today. And if it does exist, it’s an internal report that’s built by your development team, and that’s kind of the problem we’re trying to solve.

Nivi: Cool! Thanks again for spending the time talking and also sponsoring the interview with Sean and making it free for everyone.

Hiten:  Absolutely! Our pleasure.

Get more startup advice

Trent: You’ve been listening to Hiten Shah, from KISSmetrics.com on how to optimize your web apps. And if you want to find out more, you can check out the KISSmetrics.com website. You can also follow Hiten on Twitter @hnshah, or check out his personal site at hitenshah.name.

Be sure to also have a listen to Nivi’s interview with Hiten regarding Survey.io, and the two-part interview with Sean Ellis on bringing your product to market, on the VentureHacks.com website.

Of course, if you’re into all other sorts of good advice for startups, head on over to Venture Hacks and just soak it all in.

Thanks again for stopping by. See you next time!

That’s what I’ve been hearing since we published Part 1 of our rare interview with Sean Ellis.

Here’s part 2.

In Part 1, Sean discussed what you do before product/market fit: how to get there, how to measure it, and how to survey your users so you can improve fit.

In Part 2, he explains what you do after fit: optimizing your positioning, implementing a business model, and optimizing your funnel — all so you’re prepared to acquire users quickly and profitably.

If you don’t know Sean from his blog or tweets, he lead marketing from launch to IPO filing at LogMeIn and Uproar. His firm, 12in6, then worked with Xobni (Khosla), Dropbox (Sequoia), Eventbrite (Sequoia), Grockit (Benchmark)… the list goes on. 12in6 “helps startups unlock their full growth potential by focusing on the core value perceived by their most passionate users.”

This is the first time Sean has done an interview on the record. I’m really psyched he’s making his insights public — this interview is a must-listen.

SlideShare: How to bring a product to market, Part 2
Audio: Interview with chapters (for iPod, iPhone, iTunes)
Audio: Interview without chapters (MP3, works anywhere)
Transcript with highlights: Below

This inteview is free — thanks to KISSmetrics

We’re bringing this interview to you free, thanks to our sponsor KISSmetrics.

Sean is an advisor at KISSmetrics and we interviewed their CEO, Hiten Shah, in How to measure product/market fit with survey.io. KISSmetrics built survey.io with Sean — now they’re collaborating on KISSMetrics, a new tool for funnel optimization that we’ll discuss in an upcoming interview with Hiten Shah.

Prerequisites

You’ll get more out of this interview if you also read:

1. Part 1 of our interview with Sean.
2. The startup pyramid.
3. We use several phrases interchangeably in the interview: Growth = scaling = acquiring customers with a known ROI. Preparing for growth = after product/market fit = optimizing promise + implementing economics + optimizing the funnel.

Outline

Here’s an outline and transcript of Part 2.

1. What comes after fit?
2. Prepare for growth
3. a) Put the metrics in place
4. b) Optimize the funnel
5. c) Optimize the messaging
6. Prepare for growth as quickly as possible
7. Remove bottlenecks to preparing for growth
8. Preparing for growth is not a low-burn period
9. Preparing for growth doesn’t require growth
10. Use a business model to grow quickly
11. Grow and create new channels
12. Nail the first user experience while preparing for growth
13. Leave no room for the competition
14. Marketplaces are an exception to this model
15. Why Sean decided to focus on startup marketing
16. Just scratching the surface

Transcript

Music: The Equator by Tortoise

What comes after fit?

Nivi: Right. One thing you talk about is, once you have the product/market fit, trying to get through the next few steps as quickly as possible. So it would be great if you could talk about that, and also, for people that don’t know what the pyramid looks like, what are the next few steps?

Sean: Sure. As I said, if you don’t have product/market fit, you want to be obsessively focused on getting there, however you’re defining product/market fit. I think the easiest kind of definition to work toward is trying to get 40% of your users to say they’d be very disappointed without it.

And during that period you’re trying to stay very low burn and very conservative on all of your expenses within the business. And once you find 40% of your users that say they’d be very disappointed without the product, then you’re in a position that you have a business that can grow now. So then the question is: do you try to grow the business right then or are there some things that you can still do that are going to make it even easier to grow when you’re focused on that? And what I’ve found in working with so many companies at this [post-fit] stage is that you’re definitely better off waiting a little time until you try to grow the business.

Nivi: And what does it mean to grow the business? You’re talking about spending money to acquire users, essentially.

Sean: Acquiring users, for a lot of businesses, means starting to spend money, but it might also mean really trying to crank up the virality of the business, or it might be SEO that doesn’t require a lot of spending.

Nivi: But it’s time and money that you require for people… [interrupted]

Sean: Yeah, and focus. The time piece, you could say, has time or focus. But it’s essentially saying…

Prepare for growth

Sean: Is now the time to orient — especially the CEO and the marketing group and maybe a sales group — is now the time to build up that group and just say, let’s step on the accelerator and do everything that we can to get very aggressive about growing this business? And I would say, not quite.

The reason being, one, up until this point you shouldn’t have really focused on really trying to get the metrics into your business, so one of the ways that you can effectively grow a business today is, you can experiment in lots of different areas and figure out, in which areas do I invest a dollar and get that dollar paid back the most quickly or make a really strong return on the investment? But you can’t do that if you don’t have the measurement systems in place. So one of the first steps that you want to do is specify what metrics you really need to be tracking in order to be able to grow this business. And you want to then work to implement those.

So, to some degree, Google Analytics can help you, but the problem with Google Analytics is that you don’t really have tracking on a very user-specific level. So maybe you can track with Google Analytics or with Website Optimizer, even with Google AdWords with action tags, you can track a first transaction through dollars spent, but what you can’t track is lifetime value. And ultimately, expected lifetime value is what you should be basing your customer acquisition investments on.

Nivi: With the per-user tracking, is that what KISSmetrics is for?

Sean: That’s what KISSmetrics is working on. For other companies that I’ve worked with, it’s basically been database driven reports where we’re cookie-ing users when they’re coming in to track them back to the source, and we’re recording that information in the user’s permanent record in the database, and that way any revenue that’s generated from that user, we’re able to tie back to the money that was spent on that referring origin, coming in.

The reason why I started working closely with KISSmetrics is that all of my projects were six months, and that type of a system that I just described tends to be fairly complicated, especially as you start to track where you’re losing users along the way, and some of the other pieces. It was taking a big chunk of the time that we had together to implement that, and there were a lot of bottlenecks around it, so I was trying to find someone who could build that type of a solution as an off-the-shelf type solution and KISSmetrics was on their way to doing something like that. So I’ve been advising them for a couple of years to try to get the full vision out there of something that can really give the metrics that any business needs to be able to drive and manage growth.

Optimize the funnel

Sean: So metrics is just one part of it. Obviously, as I started to touch on right there a second ago, most companies initially have a very inefficient customer-acquisition process. So it’s not just how effectively can you spend money externally, but how can you convert people once they get to your website to having a gratifying first experience, and then ultimately buying your product? And metrics are a good way of determining experiments that you run along the way — how can I get more and more people to actually convert?

That can be through landing-page testing. You can be really effective on that front, or all the way down to just testing purchase prompts. There are all different areas, so you want to have a pretty sophisticated measurement system in place to do that.

I’ve worked with businesses where, at the start of that process to the end of that process, in a matter of just a couple of months, we’ve seen five times as many people purchasing just by more efficiently on-ramping them into the product and converting them into being paying customers, which means that when we then focused on trying to grow the business and buy growth, we could spend five times as much money to get someone to the website at the same return on investment, after that process.

And when you can spend five times as much money to get someone to the website, there are a lot more viable marketing channels that are open to you — a lot of times channels that would not have been effective at all before. If you’re putting a dollar in and you’re only getting 50 cents back, after you’ve gone through this process and you try that channel again and now you’re putting a dollar in and you’re getting $2.50 back, you’re going to put as many dollars as possible into that channel, and you would have cut it previously.

Nivi: And the expected LTV of the user is equal to your allowable acquisition cost per user. Is that right?

Sean: That would be your allowable acquisition cost for a break even. It kind of depends on what your objectives are. If you’re trying to do market share you might even be buying users at a loss initially. If you’re trying to generate a profit, you probably want to build in some profit on that.

Optimize the messaging

Nivi: Right. So, we put the testing in place. What other steps are we going to do before we grow?

Sean: So messaging. You want to make sure that you’ve got good messaging. So part of that was to get the product/market fit. You might be doing some positioning and messaging. But this is where you want to do a lot of landing page optimization and fine tuning just to see, what’s the best way to convert users coming in?

Nivi: In terms of positioning.

Sean: Ultimately in terms of converting them, positioning is part of that. To me, positioning is sort of the qualitative side and then testing and optimization is the quantitative way of getting to the best result. Hopefully, where you end up is somewhat consistent with what your positioning usage led you.

Prepare for growth as quickly as possible

Nivi: And what are your thoughts on just going through this whole process as quickly as possible?

Sean: That’s a good question. The mistake that I’ve seen a lot of companies make, particularly those that struggled to get to product/market fit, is that the whole mentality while you’re trying to get to product/market fit is to be super conservative and not spend very much money, so a lot of companies kind of [say]: OK, we got there. Now let’s prepare to grow.

And they, again, are really conservative and they’re taking six months to…. A prime example would be: I need to spend $100 a day or $200 a day to know that this is the best-performing landing page, or home page even, or I need to send 1,000 people or 10,000 people through there, and most people will space it out over two weeks. They’ll say they don’t want to spend more than $100 a day while we figure that out, so we’ll space it out over two weeks.

In reality, if you have some place where you can spend the money — let’s say you’re doing that through a Google campaign and you’re not getting the cap on that, but you’re saying, I just don’t want to spend more than $100 a day, you’re essentially going to spend exactly the same amount of money to get to the answer if that’s a good page or not. One way it’s going to take you two weeks; one way is going to take you one day. If you consider that time is money as well, then you’re closer to being able to accelerate the business after one day than you would have been otherwise and it costs you the same. But it’s just really hard I think, a lot of times, for people to go through that mentality.

Remove bottlenecks to preparing for growth

Sean: Another example would be if you are tight on graphic design resources internally, if you test 50 landing pages or 100 landing pages or 1,000 landing pages, you’re going to be so much more efficient at being able to drive growth in your business that it just doesn’t make sense, if your big bottleneck is on graphic design, then you want to bring in two or three people. Pay them a little bit more. It’s a temporary spike in costs to get much faster to the point where you’ve got a lot more landing pages in there.

So you combine those two things. We’re going to spend more to get people through, and we’re going to be able to test five landing pages at a time rather than just two. So being able to do all of those things… The goal should really be….

Preparing for growth is not a low-burn period

Sean: You have two low-burn periods in the business. The first is pre-product/market fit. You’re trying to spend the least amount possible.

And then the second low-burn period is once you accelerate the business you’re working within the parameters of your allowable acquisition costs, so you can actually go to being cash-flow positive pretty quickly at that point.

One of the companies that I worked with, once we got through this period of — it took us about four months to transition to being able to really accelerate the business and we got it to the point where we could scale marketing to close to a million dollars a month with a fast payback on those marketing dollars.

And at that point we had to raise some more money to be able to fund those campaigns. We raised the money on some very proven metrics and went cash-flow positive the month after we raised the money. So it shows that if we had taken 16 months to go through and really optimize that experience, it probably would have taken a lot longer to get the cash flow positive, but we were very focused and aggressive on it through that period of time and could really accelerate that business in a very cash-efficient way once we got through that period.

Preparing for growth doesn’t require growth

Sean: The mistake that most people make is they’re trying to manage growth and optimization at the same time. For this period of time, growth isn’t important. That’s the other thing that’s really hard for people to consider. That is not the objective — efficiency is the objective.

So for the first part of the pyramid, the base of the pyramid, you’re just trying to create a product that’s good enough, that people want, so growth is definitely not important.

A lot of companies still pat themselves on the back if they grow faster than expected through that period. You shouldn’t, because what happens, especially if you go to your VCs and you pat yourself on the back publicly to them, then you’re essentially saying growth is important to us, and you’re setting expectations that they should want more of it. So basically, say: Hey, we grew. That’s not what we were trying to do yet, but it’s positive on the hope for the business, but before we really focus on growth we want to get our product/market fit number to this.

Now, in this case, basically we want to be able to grow as efficiently as possible when we’re focused on it. And so what we want to do is build in efficiency in the business and the customer acquisition monetization.

Use a business model to grow quickly

Sean: So this is the time when you put in the business model as well, because you can’t spend real aggressively if you aren’t confident that you’re going to get that return on the investment.

A lot of companies make the mistake of… at least I believe it’s a mistake. This is probably one of the most debatable things that I say; a lot of companies believe they can grow faster without a business model. To me, having seen companies that can arbitrage dollars to grow and do it really effectively, you can just accelerate the business so much with that, why would you not spend money to grow? But you can’t do that if you don’t have a business model in place. So that’s why I tend to want to put a business model in place at that point.

The mistake of trying to put a business model in earlier, before you have product/market fit, is that you don’t know if you’re charging for the right things in the right way. You may be charging for things that are totally irrelevant for people and giving away stuff that is really important to people. So that’s one of the big reasons I like to delay the business model until, one, I have product/market fit, and two, I know why I have product/market fit — I know why people love the product.

Grow and create new channels

Sean: Now I’m in a position to put the business model in place, work on all the efficiency on the customer acquisition side, and do everything so that when it comes time to focus on growth, the marketing group or the CEO or whoever is going to be managing that process can be completely focused on driving growth — they’re experimenting with lots of channels and finding those that work and killing those that don’t, and basically just as quickly as they can, adding as many new customer acquisition sources to the mix.

And you shouldn’t focus on that until you know how to efficiently convert people and until you have a product that people really need. So that’s why this pyramid just gives you the idea that until you have really achieved that level, and if you’re trying to do multiple things at the same time, you’re not going to give things the necessary focus to actually accomplish them.

And your goal is ultimately to be growing a product that has very efficient conversion, that you’ve got all of the metrics in place to really manage that growth, you’ve got a product that people love, and at that point you’re starting to become a regular company. You’re organized into product teams and marketing and sales and finance and all of those things that are needed to create an efficient, high-growth company that can manage that growth and not implode under that growth.

That’s really why in that center period of time, it’s exciting. What should be driving you, day in and day out, is that we’re close. We’re right to the point where we’re almost going to accelerate the business.

Nail the first user experience while preparing for growth

Sean: So, one other piece that I think fits into this is that a lot of times it requires a lot of experimenting with how you introduce people to your product. What do you show them first? What is that whole first user experience?

The mistake a lot of companies make is that they try to get that first user experience right before they have product/market fit, so those companies are essentially pulling limited resources off of their core product and saying: we’re going to focus those resources on getting that first user experience right. Then that means that they’re not really dealing with the problem. They can maybe get twice as many people in to experience the product, but it still sucks. So they’re not really dealing with the issue.

But now everybody who comes in, or a big chunk of people who come in, are saying that they’d be very disappointed without this product — there’s a lot of love for that product. At that point, basically hitting the pause button on core-product development for two months isn’t even going to hurt the business because people already love it, and taking all of those resources that would have been focused on honing that core-product development, and refocusing those on: How do you introduce people to that? What does that first user experience look like?

The mistake that a lot of companies make is that they’ll take one person off to do that, and it’s going to take them a lot longer to do the development to get that right. But now you’ve got that core-product experience right, taking the majority of your development resources to get through it in a few weeks is going to get you there faster and that is going to put huge dividends on your business.

Nivi: You’ll become master of your own destiny much more quickly. Right?

Sean: Yeah. The way that I’ve been able to get development teams excited about that is, one, I give them the context of the whole big picture, and then I essentially say this is not something that we’re going to ask you to keep coming back and refining the first user experience. Once we’ve really got it right, backed up by numbers, backed up by saying twice as many or three times as many or ten times as many people are getting in and experiencing that core-product experience, then you can go back into just working on that awesome core-product experience that you’ve created, and evolving that and just continuing to create a great product experience.

But otherwise, we’re going to have to keep asking you for help for the next six months, until we get that first user experience right. It’s just much better for everybody, for you guys, to put the pause button on that, help us get this right, and then we’re going to be able to grow the business and it’s going to be a lot more exciting.

Leave no room for the competition

Nivi: And once you’re growing the business, what do you think about growing quickly to eliminate slack in the marketplace and basically leave no room for the competition?

Sean: Once you have product/market fit, and once others see you having traction, you’re going to attract a lot of copycats into the business. And they should be trying to come in because, like I said, the product/market fit is kind of the hardest thing to get to, so once somebody sees that you have a lot of passionate users who really love it, it’s the easiest way to create a moderately successful business — to knock that off.

So you have a short-term advantage, which is you’re able to respond to all the real product feedback that you’re getting from users who are coming in and experiencing that, so you can start to evolve your product better.

So partly what you want to do is just buy every related keyword, maxed out, and be very efficient on those keywords so that you can spend a lot more money on them acquiring users. So, the next guy can’t even consider spending keywords, so you’ve killed that channel for them. You just want to basically make it so that by the time someone can respond, it’s going to take them several months to be able to come out with something that’s close to your product. By the time they come in there, there’s just no market left for them to play in — you’re really redlining that market.

It’s interesting. In companies that I’ve been with through the growth period, the products that come easiest are the ones that often get the least focus because, oh, we’re hitting the targets on those numbers. We have to work harder on these other numbers. But it’s always the products where the growth comes easiest where the competition comes in because we didn’t take the slack out of the market in those products, and then suddenly competition that we didn’t expect, and then it’s not so easy to grow any more.

Marketplaces are an exception to this model

Sean: I think all of the things that I’ve talked through are a snapshot of what I think is best right now. There are so many exceptions in the different types of businesses that are out there. For example, marketplaces do need to focus on growth a lot earlier. Their business is a function of the people that are on there and how they’re using the product, and the experience changes for everybody else with more people on the product. eBay would be a good example of that.

I think each of these things are some of the guidance that I wish I had in the startups that I was in, but don’t just take it exactly as-is and think you can just plug it right on your business and that it will work. You need to sort of interpret what this means for your business.

Nivi: Do you have any experience applying any of this theory to marketplaces at all?

Sean: Yeah, Y Corp. is a company that I’ve worked with that is a marketplace that I’ve definitely learned a lot and learned that it’s different than a lot of the companies, like Dropbox or Xobni, where it’s a more specific product that, with a million users or fifty users, there would be experience where the product doesn’t change that much, where for Y Corp. the experience changed a lot.

Nivi: So how do you still take a thoughtful approach when you’re building a marketplace?

Sean: It’s just a lot more complicated. I think, for me, one of the things that’s kind of come out, in my head, on marketplaces is that you probably need deeper pockets for it. You don’t have the luxury of focusing quite as laser focused on each of the steps in the pyramid that I talked about — that you actually do need to think about growth earlier on, and that even when you’re then focused on growth you need to make sure that you’re constantly finding out if the love is moving, if it’s on something else. It’s just a lot more dynamic to try. It’s fun. It’s interesting, but it’s definitely harder.

And I’ve had enough conversations with other people from dating sites to anything that’s got the experience based on the mix of people that are there, that it’s really something that affects any businesses like those. So that’s one piece that, don’t just feel like you can plug this in directly. There are probably exceptions for a lot of these things.

Why Sean decided to focus on startup marketing

Nivi: Why did you decide to focus on startup marketing, and why is it important to you and how did you come to that conclusion?

Sean: It actually happened when I was coming off of LogMeIn and figuring out what I was going to do after LogMeIn. I knew that the company had gotten big enough that it was not really the size that I wanted to be working in anymore, and that I wanted to go back to early stage.

So it was more just thinking, OK, I can jump into another startup and hopefully I can be fortunate enough to get into a startup where it has the success. But just looking at realistic startup success rates, LogMeIn was the second in a row of two startups that I had run marketing from launch through a NASDAQ IPO filing, and I just knew, realistically, that there was not a good chance that I was going to be able to do that with a third one.

And so partly what I was looking at was just in general…. I kind of went through the thought process of: did I just get completely lucky, which certainly had something to do with it, or were there some things that we did that actually led to that success? And when I really started thinking through both of the companies, I came to the conclusion that the hardest part and the part that really mattered in building a successful company, was what we did in the first year, and really just figuring out everything about the business: Who really needs this product? Why do they need it? How are we going to charge for it? How are we going to acquire new users?

All of these things that needed to be figured out, once we had them figured out then it was much more about, OK, now how do we get more users? But a lot of the moving parts had stabilized at that point and we could really focus on just getting more new users. And so I thought, if that’s really the most important part, I thought that it was also the most challenging part, and so future success that I have is really going to be based on how well I can do that first part.

And then I realized that in 10 years, across these two startups, I’d only spent one or two years in that really challenging stage, so if I really wanted to get good at that I was going to need to figure out how to work in that stage a lot.

And the problem with startups is that every person working in a startup full-time, basically has an option vesting period that’s usually about four years on those initial options. So you work really hard in the beginning and you get some level of success and then it takes another four years to be compensated for that success, so there’s no way you’re going to create all this success and then leave the company right away.

So for me it was more about: how can I work really hard in that stage and actually be compensated for it? The only thing that I really figured would be the best way to structure it would be as an interim VP marketing role, and I was introduced, from one of the investors in LogMeIn, to a company that was in his portfolio called Xobni. They were still in private beta, and he said just talk to the guys. So we had a conversation that ended up extending for a couple of hours and we really bonded over some things.

And I got excited about the product and they got excited about potentially working with me and tried to bring me in long-term, full-time, and that was not something I wanted to jump right into. I was really passionate about figuring out this early stage and doing cycles there. So we decided that it made sense to pursue an interim VP marketing role. So I basically went there and thought, I’ll do interim VP marketing role after interim VP marketing role, for the next several years — essentially negotiate my exit before I came in so that I’d be able to keep a reasonable amount of options. That was the initial plan, and the whole idea was just to get the experience, up front, and practice, up front, to get good at that and to really document what’s important and what’s not important.

I think the challenge that I figured out was that, really, on the marketing side, there’s a lot of bottlenecks to get a lot of the things done, like optimization and trying to get resources to do a lot of those things. Everybody’s scrambling to get the overall product out.

So I knew a lot of what needed to be done, but there’s just a lot of waiting around until you have the tools that you need to do that. So where I’m looking at everyone around me working 14 hours a day, and wanting to put that time in myself, there just wasn’t that much to do. So I almost felt guilty that I wasn’t putting in the same amount of effort that everybody else was, so I still spent the time in the office, but I just found myself doing a lot of things that maybe weren’t that important.

That was why, after Xobni, I decided that I was going to do a more leveraged model where I could work with a couple of companies where they actually had somebody inside to manage the execution of things, but that I would actually be more on the outside, being able to not spend more time in the company than I needed to spend. So that’s why I did Dropbox and Eventbrite in that sort of way, where I was balancing two companies at once, and that worked pretty well.

Just scratching the surface

Sean: I feel like I’m still just scratching the surface on what can be learned in early-stage startups. There are not a lot of people that have deep enough experience and frequent enough experience in that really critical up-front stage to get good at it. It’s been fun to try and figure it out, but there’s a big variety of companies. There are certain things that I think are applicable to all businesses.

One of the most important discoveries in the last couple of years, since I’ve been focused on this stage, is figuring out this very-disappointed number and just understanding that trying to grow a business that doesn’t have users that would be disappointed without it, or don’t consider the product a must-have, is going to be a difficult and frustrating experience for anybody, including me. It’s humbling. I can’t do it.

I can’t do it very well, and when I see companies doing it I have a lot of admiration for them to be able to grow a business that is weak on that [fit] number. Those are the hard-core marketers that are kind of too good for their own good, because they’ve basically allowed the business not to focus on what the core issue is that’s holding them back. But those people would really benefit from taking a step back and tightening up some of these things.

I’m doing an experiment with Webs.com, for example, that is a later-stage company. They’ve been around for a long time. They’re a top-100 website and have a ton of user passion and a lot of the things that I’ve done to help prepare companies to grow are things that Webs hadn’t done, so we’re working together to see if we can retroactively put some of these things in place to accelerate their business.

And for me it’s all about learning and just experimenting in companies to find out what’s unique to startups and what’s unique to later-stage companies and what applies to everything.

Nivi: Great! That’s great. I just want to say thanks very much, Sean, for talking to us.

iTunes users: Click here to subscribe to the podcast
Everyone else: feeds.venturehacks.com/venturehacks

Put the podcast on your iPod and listen at the gym, in the car, on the train. If you’ve been skipping class, catch up on all the interviews in our Podcast category.

We’re going to try to do more interviews in 2k10. Hopefully with guys like Don Reinertsen, Al Ries and Jack Trout, and Steve Blank.

Thanks to KISSmetrics for supporting our interview with Sean Ellis. If you want an intro to KISSmetrics, send me an email. I’ll put you in touch if there’s a fit. Thanks. – Nivi

Hiten Shah from KISSmetrics recently sat down with me to explain how to use their product, survey.io, to measure product/market fit and find the “best grass” in your product. You may know Hiten from his Crazy Egg days.

survey.io is a free survey tool that helps you implement some of Sean Ellis’ techniques to get to fit. KISSmetrics actually built survey.io with Sean. Unfortunately, there hasn’t been great documentation for survey.io besides Sean’s launch post. Until now.

SlideShare: How to measure product/market fit
Audio: Interview with chapters (for iPod, iPhone, iTunes)
Audio: Interview without chapters (MP3, works anywhere)
Transcript with highlights: Below

Prerequisites

You’ll get more out of this interview if you also read:

1. An example survey from survey.io.
2. Our interview with Sean Ellis.

Outline

Here’s an outline and transcript of the interview. The interview and transcript are about 19 minutes long so we’ve highlighted some of the juicy bits to get you started.

1. survey.io: Before product/market fit
2. survey.io measures fit
3. How did you discover the product?
4. How would you feel if you could no longer use the product?
5. What would you likely use as an alternative if product were no longer available?
6. What is the primary benefit that you have received from the product?
7. survey.io is open-ended
8. The flock will always find the best grass
9. You don’t need survey.io to find the best grass
10. Have you recommended the product to anyone?
11. What type of person do you think would benefit most from the product?
12. How can we improve the product to better meet your needs?
13. survey.io is more powerful with filtering
14. Would it be okay if we followed up by email to request a clarification to one or more of your responses?
15. Upcoming survey.io features
16. Get qualitative feedback before fit
17. Must-have % by industry
18. Ask the must-have question

Transcript

Nivi: Hi there! This is Nivi from Venture Hacks, and I’m here with Hiten Shah from KISSmetrics.

First of all, I want to thank them for sponsoring our interview with Sean Ellis and making it available, free, to you guys.

We’re going to talk a little bit about what KISSmetrics does and how it ties into the things that we discussed in the interview with Sean. And Sean is actually an advisor to KISSmetrics. Is that right?

Hiten Shah: Yes, that’s right.

survey.io: Before product/market fit

Nivi: Cool. The interview with Sean was broken into two parts — before product/market fit, and then after product/market fit, where you do the things to prepare for growth.

Why don’t we talk a little bit, first of all, about what you guys have and do for the before product/market fit stage?

Hiten: Sure.

Nivi: Take it away!

Hiten: Yeah, I will! First of all, I want to thank Venture Hacks. We’re happy to support it and thank you guys for bringing interviews like Sean and Eric and all those guys to a bigger audience. I think you guys are doing a great job of that, and that’s why we’re pretty happy to sponsor the podcasts.

In terms of pre-product/market fit we’re following a lot of these practices in our own business, so we’re eating our own dog food, so to speak — everyone says that we’re actually really doing it.

survey.io measures fit

Hiten: So that’s why we built survey.io with Sean Ellis, and the idea of it was, Sean’s got a bunch of questions he asks. The key question is: how disappointed would you be if a product name, say, Google didn’t exist? There are a bunch of key questions there. I’m sure it’s gone over in the interview, previously, in the first part.

But basically, there wasn’t a template that had that question on it. So we worked with Sean Ellis to come up with a simple product, a simple tool that was free that people can use to assess whether they have hit product/market fit, and that’s what we did with survey.io.

So anyone can go into survey.io and basically send off the survey to their users and get an idea of whether they have a product/market fit.

1. How did you discover the product?

Nivi: So the main questions are: How did you discover the app?

2. How would you feel if you could no longer use the product?

Nivi: How would you feel if you could no longer use the app — which is the key product/market fit question — very disappointed, somewhat disappointed, or not disappointed at all?

3. What would you likely use as an alternative if the product were no longer available?

Nivi: What would you use as an alternative?

What’s the thinking behind that? Do you have any thoughts on that?

Hiten: Yeah. It helps you understand how you fit against competitors, or what space you fit into. You might think that you’re a customer feedback tool, while customers might weigh you as a customer support tool. So that question is actually pretty telling in helping you understand, from your customer’s perspective, what other products they would compare your product with. So I think actually that’s a pretty important question.

Nivi: Have you ever seen any surprises on that, just to flesh it out, either with you guys or other guys?

Hiten: Yes, actually I have. In a few early questionnaires about the KISSmetrics product, an older version of it, people actually put us up against optimization tools like A/B testing tools and things like that, and we didn’t have any functionality around that. So I think that was an interesting insight. Maybe it was just the fact that the analytics space is pretty crowded and there are a lot of tools in it and it’s kind of a nebulous definition, like what is analytics?

4. What is the primary benefit that you have received from the product?

Nivi: The next question in survey.io is: what is the primary benefit that you’ve received from the product?

What’s the thinking behind that?

Hiten: The thinking around that is, basically, if you understand what benefits people see your product for and you filter it by people that would be very disappointed if your product didn’t exist, you can get a really good idea of what passionate customers consider the benefit of your product — and in their own words. So you can use that to change your messaging, and kind of A/B test a bunch of things around landing pages and things like that and see if you can have a higher conversion rate based on the messaging you have there.

survey.io is open-ended

Hiten: I’m thinking that this interview with Sean has probably had a strong impact on getting people to understand what these questions are all about, much more. And that’s what I’m hoping happens because I get a lot of questions about all of these things, as well: What does each question mean? What do I do with these answers?

And that’s actually a typical problem with most survey tools, as well. If you notice, in this survey there aren’t any questions that are very least, most, least, and have radio buttons when you go there, because you’re basically making your customer think a lot with those types of questions. With these open-ended questions people are just typing in whatever they want to say, and then it’s up to you as the person giving the survey out, to analyze the results. So, that’s a big difference with this survey and the questions that are asked.

The flock will always find the best grass

Nivi: Right. Going back to the question: what is the primary benefit you’ve received from the product? Sean quotes Vinod Khosla a lot that, “the flock always finds the best grass.” I didn’t actually understand what he ever meant by that until the interview where he… What he really means is, look at the must-have users and see where in the product they consider the grass to be — they’ve found the grass. Before that, I didn’t even really understand what Vinod meant by it. I guess that’s what he means by it. Right?

You don’t need survey.io to find the best grass

Hiten: Yeah. Let me give a different perspective. I think that’s absolutely correct, but let’s say you don’t know about product/market fit, and you don’t know about giving out this survey or asking that question because you’ve never seen Sean’s stuff. Someone who has a lot of customers, and only some of them are addicted to the product, what they typically do is go look in their logs or look in their user database and see who’s logging in the most or see who sends support requests or is contacting the company the most, and they would consider those people the flock that’s going towards being very passionate about the product.

So I think there are number of ways you can determine this type of stuff. So to me, what Vinod Khosla said feels like there are multiple ways you can determine this. This seems to be a very strong and efficient way compared to how people used to do it.

In some of our own products, in the past, we would just do it based on: Oh! This guy logs in every day. Or, this guy keeps bugging us and wants features, and at the end of every email says, “Oh, I love your product!” We’re actually getting that with our latest product that we have, and those are the people who typically you try to cater to and things like that. This puts a little more science behind it.

Nivi: Right. And this tells you why they love the product.

Hiten: Absolutely.

5. Have you recommended the product to anyone?

Nivi: The next question is: Have you recommended the product to anyone? And if so, please explain how you described it.

What’s the thinking behind that?

Hiten: This goes back to something called Net Promoter Score.

These questions are all developed by Sean, but this question is interesting because when I first saw it, it reminded me of Net Promoter Score. Net Promoter Score is a methodology that a lot of Fortune 500 companies have implemented, where they ask people: how likely are you to recommend this product? And they have a scoring system based on the responses.

Nivi: And it’s a scale of 1 to 10.

Hiten: Yeah, it’s a scale. Exactly. It’s not a simple question. It makes people think. But they have scoring around it, so I don’t know exactly how the scoring is. But if the average people say 3 to 4, and then you make some changes, you want to see those people start saying 5 to 6, or whatever it may be. And then you group… It’s not a gauge….

Nivi: Do you take cohorts of those?

Hiten: Yeah, you take cohorts of those people and try to improve them over time. And there’s a whole methodology around it. It’s pretty complex. Another part of it is….

The key thing about this question that people don’t get at first glance is that they think it’s really asking them if they’d recommend, and trying to make them recommend to people, or trying to suggest they should recommend it, and it’s actually not about that, and neither is Net Promoter Score. All it is, is that there is some psychology around if you’re willing to say you’d recommend it. You would be bucketed into more of a passionate customer. It doesn’t mean necessarily that you’ve ever recommended it, or that you would, but it’s more just to solicit how satisfied customers are with your product.

So this question doesn’t come out of NPS, because Sean didn’t even know about Net Promoter Score, I believe, when he created it, but it’s got the same kind of feeling towards it. So it’s just basically another identifier of people who are passionate.

But the real thing about asking them the yes or no question is when you ask them to describe how they would describe it to a friend, because in this case it’s an open-ended field and they’re typing in the way that they would describe it, which is basically another gauge, a different sort of angle on: what benefit have you received from the product? You’re having them actually help you figure out what your messaging should be, quite frankly.

Nivi: Yeah, and that is, literally the positioning of the product, the position it holds in the user’s mind, and that’s how he would describe the product to someone else.

Hiten: Absolutely. Yep, it’s really powerful.

6. What type of person do you think would benefit most from the product?

Nivi: Just this next quick question: what type of person do you think would benefit most from the product? Question 6.

Hiten: Yeah, I actually have a term that relates to this question a little bit, but my term is data porn. Some things are just data porn. This doesn’t fall into this category, but it just reminds me of that because I get a lot of enjoyment out of this question when I look at results, because people will describe it in all sorts of ways. You’ll hear people say: my best friend would best use it.

You know, depending on what type of product, obviously. I think this question is important because it helps you understand what your target market should be, potentially. Right?

Nivi: Right.

Hiten: And a lot of times people won’t even say it’s someone like them. They might say that it’s someone different than them, and that could give you some insight into, based on your current messaging, what people really think the type of person who would be attracted to your product would be.

So, I haven’t learned as much from this question, but it’s always fun to look at, because it’s another one of customers saying something.

Nivi: Just thinking about the interview with Sean, which you haven’t heard yet, I think this could be helpful, in particular, when the product is very horizontal and you’re trying to figure out which vertical in that group is really the one that you’re going to go after, potentially. For example, Sean talked a little bit about how they initially positioned the product in a feature-based way so as not to narrow down the use case.

Hiten: Yup.

Nivi: So here you’re basically asking them the title of the person who’s going to use the product, or what the use case is.

Hiten: Yeah. Yeah.

Nivi: So I think it can help you narrow down the positioning and the market that you’re going to go after.

Hiten: That’s a great point.

7. How can we improve the product to better meet your needs?

Nivi: How can we improve the application or product to better meet your needs? Question 7.

Hiten: That’s always a great question. People tell you…

This one’s interesting. I think this one gets the most quantity, especially with passionate people. They want to tell you exactly how you can improve their product.

If you haven’t asked this question to your customers, I don’t understand why not, because they’re going to give you all kinds of insights into little tidbits they don’t like. I’ve seen people answer this question and go into: oh, the button should be moved; or the UI is like this; or even try to get into really descriptive stuff.

survey.io is more powerful with filtering

So it’s just a cool question, and again, all these questions are really powerful when you start filtering. That’s why actually we don’t have an export feature that’s public on this thing, but a tidbit is that if you just add a /export to your results, you’ll get an export of the data.

Nivi: OK, nice. Yeah, and the filtering would actually be super interesting. The first segment filter is by must-have, nice-to-have, and I-don’t-care users.

Going back to Question 7: how can we improve the product? Again, you haven’t heard the interview yet, but what Sean talks about is going to the nice-to-have users and figuring out how you can improve the product for them and turn them into must-haves.

Hiten: Yup.

Nivi: So he gave the PayPal example, where some of the nice-to-have users may have still been using the mobile to mobile payment. You don’t care about those guys because you’ve seen like 1,000x increase on desktop to desktop payment. But of those nice-to-have users who are using the desktop to desktop payments, you might want to know how they want to see the product improve; for example, better security or if it had my address book in there already or something like that.

Hiten: Yup.

8. Would it be okay if we followed up by email to request a clarification to one or more of your responses?

And then: would it be OK if we followed up by email to request a clarification? Question 8. What’s the thinking behind that?

Hiten: These are people that you have… you know what they answered, and you can segment into different groups and you can follow up with them with follow-up surveys and things like that. It’s always good to get email addresses.

What we’ve noticed, on the weakest products, the ones that aren’t as sticky, maybe are more consumer oriented, we’ll still see 50% of the people giving an email address. And you might not even have the email address of these people because one of the things we let you do with the survey is embed it on the page, and it might not be a logged-in user so you might not even know what their email address is.

So now all of a sudden you’ve got these people, you’ve got their email address, and you’ve got all these answers associated with that email address. So that just becomes really powerful to follow up with them or, back to the previous question, if you add a feature that they want, you can go notify them and let them know you added it because they wanted it. Customers get delighted by that. I don’t see why every company isn’t trying to delight their customers — it’s one of those things.

Nivi: Right. And when you’re in the trying to get to product/market fit stage, getting a list of potential, early evangelists is probably at the top of your to-do list. Right?

Hiten: Absolutely. Yeah, we don’t even let you edit the questions yet. That’s a feature coming out. We’re building a much deeper product out of this. We have some ideas around it. We’re still working with Sean on it.

Upcoming survey.io features

Nivi: What’s coming in this? The segmentation would be cool.

Hiten: Yeah. It’s been a very interesting conversation at our own company about what we should do about this. One thing is we want to, obviously, have this same thing for free, but we’ve also got some ideas on how to get qualitative feedback from customers in a much leaner way where you almost ask them one question at a time. You might build up some of these answers over time. So we’re working on embeddable surveys and things like that, to do that, and this is stuff we really haven’t seen anywhere else.

Also there are things like, if you know someone’s username or email address, you could already pass in to the survey so that you don’t have to ask for it. And then you can associate a specific customer with the response, and stuff like that, which SurveyMonkey and other tools currently make a little bit tedious to do, or basically impossible in some cases.

We don’t want to compete with SurveyMonkey or the survey tools, so we’ve really thought long and hard about what we want to do about this. We will be launching something, probably by mid-to-end-of January that’s a whole revamp on this tool. It makes it so, hopefully, you don’t have to copy and paste the questions into other tools. So editing, filtering — we’re really focused on embeddable surveys. These are just some of the pains that we’ve noticed about getting feedback from our own users.

If you ask them one question within the interface, it’s really powerful because you can get a lot more context. So in our system, if they’re looking at an analytics report, we might want to ask them: is this report useful to you? And if they say yes, say great, thanks! We love you. If they say no, we can pop-up a form field and ask how we can improve it for them. Those are the kinds of more mini-surveys and mini-questions that we don’t really see a product online that does that, so we’re going to move the product in that direction.

We’ll always have this specific survey for free though, and add some of that stuff. And this is all about….

Get qualitative feedback before fit

We feel that before product/market fit, what’s really important is those early customers and getting as much feedback as you can that’s qualitative from them, and quantitative metrics aren’t as important as the qualitative metrics at that time.

Must-have % by industry

I’m amazed by some of the percentages I see on the disappointed score that’s segmented by industry. In general, trends-wise, if it’s a SaaS sort of product, like a web app, I tend to see a higher score on the percentage of people that would be very disappointed.

Nivi: So that would actually be a great little study for you guys to publish. Right?

Hiten: Yeah. And another thing, on the opposite end, is that we notice score in the 20% to 30% for consumer applications. Typically they’re very low. People wouldn’t say that about Facebook, necessarily, but a lot of people would say that about more trivial consumer sites. I would say that you would probably see a lower score on something like YouTube, where it’s very passive and people aren’t necessarily very passionate about it and there are a lot of alternatives and there isn’t really some sticky glue that makes people really want to come back. So those scores are usually under 40%, while SaaS apps I’ve seen as high as 80%

Nivi: Well that makes sense, especially if you’re going to filter by… SaaS guys are already paying for the product; they’re probably going to consider it to be a must-have.

Hiten: Yeah, and even some of those freemium businesses though. Think about the emotional attachment to a free product you have if you’re constantly using it. You’re going to be disappointed if it doesn’t exist. Right? So I think those are the trends that we see, overall.

Ask the must-have question

Nivi: The key question here, if you’re going to walk away with one thing from survey.io, is that you should probably survey the people who actually use the product and ask them how disappointed would they be if they could not use the product anymore.

Hiten: Yup! Absolutely.

To be continued in part two of our interview with Hiten Shah, where we’ll talk about their product called KISSmetrics, a new tool for optimizing funnels.

Music: Squarepusher

If you don’t know Sean from his blog or tweets, he lead marketing from launch to IPO filing at LogMeIn and Uproar. His firm, 12in6, then worked with Xobni (Khosla), Dropbox (Sequoia), Eventbrite (Sequoia), Grockit (Benchmark)…  the list goes on. 12in6 “helps startups unlock their full growth potential by focusing on the core value perceived by their most passionate users.”

This is the first time Sean has done an interview on the record. I’m really psyched he’s making his insights public — this interview is a must-listen. We’ve broken the interview into two parts: 1) before fit and 2) after fit. This post contains Part 1 (and here’s Part 2.)

SlideShare: How to bring a product to market
Audio: Interview with chapters (for iPod, iPhone, iTunes)
Audio: Interview without chapters (MP3, works anywhere)
Transcript with highlights: Below

This inteview is free — thanks to KISSmetrics

We’re bringing this interview to you free, thanks to the kind support of KISSmetrics. Sean is an advisor at KISSmetrics and we interview their CEO, Hiten Shah, in How to measure product/market fit.

KISSmetrics built survey.io with Sean — now they’re collaborating on KISSMetrics, a new tool for funnel optimization.

Prerequisites

You’ll get more out of this interview if you also read:

1. An example of the survey.io survey Sean uses before fit. (Several phrases we use in the interview mean the same thing: Product/market fit = Fit = 40% of surveyed users consider the product a “must-have” = 40% of surveyed users would be “very disappointed” if they could no longer use the product.)
2. The startup pyramid.
3. Some of our favorite posts by Sean.

Outline

Here’s an outline and transcript of Part 1.

1. Half the marketing battle is the product
2. Must-have products make marketing much easier
3. How PayPal built a must-have product
4. Understand the must-have users
5. How to get to product/market fit
6. How not to get to fit
7. Should I launch?
8. You can’t set a deadline on fit
9. How to communicate with the board during fit
10. How to use positioning to improve fit
11. It’s frustrating to try to grow without fit
12. How many users do you need to determine fit?
13. Who do you survey?
14. What is promise?
15. Don’t over-position your product during fit
16. Pivot the business around the love
17. Focus the product on the love
18. Create a great experience around the love
19. Create a business model around the love
20. First find the love
21. Are recommendations a good indicator of fit?
22. The must-have metric is a good indicator of fit
23. Preview of Part 2: What comes after fit?

Transcript

Nivi: Hi, this is Nivi from Venture Hacks, and I’m here with Sean Ellis. We’re going to talk about how to bring a startup’s product to market.

This interview is broken up into two parts: before product/market fit and after product/market fit. Or to put it another way, what are the set of activities you have to do to get the product/market fit, and then once you’re there, what are the set of activities you have to do to prepare for sustainable growth.

I’m super psyched about this interview. This is Sean’s first-ever interview on the record. We’re going to jump right into it.

Half the marketing battle is the product

Sean Ellis: The whole idea that’s gotten me to where I am right now, where I’m doing multiple companies, is basically that I’m just trying to get lots of cycles, up front, in that really critical zone — I call it the fail zone — that if you don’t get that done right, the company will fail. And if you figure that part out, then it’s really a question of how much success is the business going to have?

Particularly, one of the things that I’ve learned that’s been a little humbling through the experience is that if nobody wants the product, it doesn’t matter how effective the marketer is, you’re going to have a really hard time being successful. And if people want the product, you don’t have to be that great a marketer — the product is just so much easier to market. So half the battle, I’ve realized over time, is just finding the right company with a product that people actually need, and that there’s a big enough market that can support growth for that business.

Must-have products make marketing much easier

Nivi: Yeah, and what might be interesting for some people to hear is what’s in that grey zone between nobody wants the product, to a lot of people consider the product to be a must-have. For example, sites that have great marketing and distribution through viral marketing, but they don’t necessarily have any real must-have-it-ness with their customers or with their users.

Sean: All right, so the question really being: the grey-zone products that fall between those must-have products, and products that maybe aren’t as strong of a must-have, but can you be successful with those?

I am just figuring this out as I go along. I know that there is no right answer on any of these things. What I do know is that if you have a must-have product, your chances of success are very high.

And the way that I figure out if a product is a must-have or not is: what percentage of people are telling me that they’d be very disappointed if they could no longer use it? But it seems like you find a lot of categories where you might say that it’s a nice-to-have, rather than a must-have.

And I think that the difference is that for a nice-to-have product — so, something where not a lot of people say they’d be very disappointed without it — that you probably can be successful, but it’s going to take a lot of effort on the marketing side to make up for those product deficiencies.

I think if you really step back and say, as a business, that ultimately our goal is to create a successful, fast growing business, that people really need our product, and that over time we have a defensible business where it’s really hard for somebody to come in and provide an alternative product where we start losing customers, that in an early stage startup you have an opportunity to put a product out there, get that initial user feedback, and then you have two choices.

If that feedback is really strong and the users say that they’d be very disappointed without it, in a large percentage, then you can try to grow the business.

Alternatively, if they come back and say that they wouldn’t be that disappointed without the product, then you have the choices where you can either try to grow it or you can decide that you’re not going to try and grow the business. And to me, my recommendation has always been to decide not to grow the business if it doesn’t have a lot of people that are real passionate about the solutions.

Actually, you can operate a startup at a pretty low cost if you basically have a small engineering team and you’re putting product out there and you’re not spending a lot to get people on there, but you’re really engaging the people that do come in. You can engage them much better if there aren’t that many people, so you’re in a position where it may take you a year or two years, but you can really, over time, start to evolve a product into being something that someone wants.

Maybe you find early on that absolutely nobody wants the product, and in that case you might want to do a complete restart on the business.

And when I’m presenting to groups, the example that I always point to, mostly so that people know that there’s hope if they don’t have great product/market fit initially….

How PayPal built a must-have product

Sean: That product/market fit is how I describe having a large percentage of people that would be very disappointed without the product. If you’re above 40% of the people saying they’d be very disappointed, I tend to say you’ve found product/market fit, and if you’re less than that, you haven’t.

So a lot of times I meet with companies that are maybe in the 15% to 20% range on that, and it’s very discouraging. You’ve worked very hard to put out a product, so the example that I give them is PayPal, where the initial product that they came out with was a cryptography product where you would use a PDA to store access codes for servers, and it was better than carrying a bunch of individual devices to access those servers. And the problem they were solving was that there was a lot of clutter in carrying those devices, and they were going to consolidate them all onto a single hand-held.

In “Founders at Work” there is a great case study on this. What Max Levchin says in “Founders at Work” is that they put the product out there and “nobody really needed it.” That was a direct quote — “nobody really needed it.” So what they decided…. Most companies would try to interpret the data and keep blasting away and say that somebody needs it. They just don’t get it yet.

Nivi: We need more distribution….

Sean: Yeah, they blame marketing, they blame sales, but ultimately, it took a lot of guts, and just honesty, to say: You know what? We created something that nobody really needs.

So what they did at that point was they stepped back. They asked what are we good at? We’re good at developing for a handheld PDA. We’re good at security. What else can we do with these skill assets that we have? And they decided that they were going to get into the mobile PDA payment space, where basically, people could beam payments back and forth.

And on the initial vision they were able to raise a couple hundred thousand dollars. On this new vision, they were able to get a prototype out on that initial money that they had raised, and they were able to raise… I think it was four million dollars. I don’t remember the exact number on it. And they actually beamed the VC money over so they had a good enough prototype to be able to accept the payment via this PDA payment. And then they were off to the races, and they started executing the business.

I think, in that case, my guess is that if they had surveyed those users who were using PDAs to make payments; it was more of a party trick. It was something that there wasn’t a real big need for, but they got users on there. They got lots of users on there, and they were able to get those users coming back. But, I’d put it in that probably-nice-to-have category.

Nice-to-have is kind of a scary thing, because you get enough success that you want to keep at it. But they were very fortunate in that they sort of stumbled into what ultimately became their market, which was, of course, the web-payment platform that PayPal is today. And that was really a function of people just starting to use it that way. And so, that’s why I focus a lot with companies on helping them figure out the use cases behind their product. If the people who have one use case consider it much more of a must-have than people who have a different use case, then that’s probably the business you’re going to be able to build a successful business on.

Understand the must-have users

Nivi: So, do you ask the must-have users what they use the product for?

Sean: I ask everybody what they use the product for, and I’m trying to see, really, who has the most passion around the product. And usually it breaks down by use cases, sometimes by user types. It’s really just looking through the initial data to really understand who really needs this product, and it can be on demographics, it can be on use cases, it can be on a lot of different things. But once you really understand a group that really needs the product, then you start to have that true North for the business — the part that you can actually start to build a business that will grow and thrive around if they represent a big enough market.

Nivi: Right. In that product/market fit stage that you’re talking about, which is basically the bottom layer of your pyramid, what don’t you do? For example: PR, trying to get a lot of distribution, increasing your burn.

Sean: Well, it depends a little bit on the type of business that you have.

So just one quick thing on wrapping up the PayPal example was, by the time they pulled the plug on their mobile-payment business they actually had, I think it was 1,000 times more people using the web payments. They had obvious traction, so they didn’t need to necessarily get that from a survey. They could just see that they had the traction there at that point. They could say: You know what? That’s where our real business is. Even though they fought it, initially, because they had different preconceptions about where their business would be.

How to get to product/market fit

Sean: I purposely try to avoid working with companies that are pre-product/market fit, because they’re in such a risk zone that, one, it’s going to hurt my reputation if I string too many of those together, and two, to ask people, who have a product that people don’t need, to pay me to help them with marketing would just not be the right thing to do. It would be a mistake for them to pay me, and it’s just not smart, from my perspective, to stake my reputation on their success.

But I found myself working with one fairly recently, on a very short, almost advisory project, of just helping them out. And my advice to them was to really be laser focused on that number and to really monitor that very-disappointed number.

So, one of the things that I’ve seen work well for companies that are in the 20% range, or even lower, of the user saying they’d be very disappointed without it, is to focus on, one, why those people would be very disappointed without it, and to really start to say, OK, this is our best signal of value that we’re creating.

And then, to look at the feedback from the people who would only be somewhat disappointed without it. Ignore the people who say they would not be disappointed without it, because they’re so far from being satisfied.

But then look at those somewhat disappointed people, and look at feedback from those people — particularly on what changes they’d like to see in the product. And look for things that relate to the very-disappointed feedback. If you take all feedback equally, you’re going to have a very broad product experience that is very disjointed.

And I think one of the mistakes when people are too responsive to user feedback is that it’s a product that does everything, because if you try to please everybody, you’re essentially going to have a very unfocused product. But if you use the feedback from your very-disappointed people to give you that focus, and now you’re looking for feedback from everyone else that relates to what the very-disappointed people have told you…

Nivi: Like with PayPal, for example.

Sean: I think, using PayPal as an example, if they had surveyed all of their users, I’m assuming that a lot of the people who were doing the hand-held payments would basically have been saying that they’re only somewhat disappointed without it. And then the people who were doing the web payments would be saying that they’d be very disappointed without it. But some of the people on the web payments would be saying somewhat-disappointed, and they would say: if only I felt more comfortable about security; or, if only I could do it faster; or, if only you could give me better records of payments that have been made.

So you could take the feedback from the somewhat-disappointed who were using hand-held payments, and act on that to try to get there, but then you’re not really honing in on where the passion is.

Or, you could take it from the other side, saying, OK, these people may be a little pickier about what defines something that’s a must-have, but the feedback that they’re giving would improve the experience for the people who already say it’s a must-have. So, it’s really trying to just hone a consistent, core-product vision that is already starting to crystallize based on the feedback from the really passionate users.

Nivi: Yeah, that’s great. So in this case, it’s related in the sense that somewhat-disappointed people are using the same segment of features in PayPal.

Sean: Yeah. It’s that use case.

Nivi: Use case.

Sean: Yeah. They’re suggesting improvements to the use case that’s consistent with the most gratified users there are.

How not to get to fit

Nivi: Great! Yeah. So that’s a great idea in terms of getting what you need to do to get product/market fit, and I’d like to talk more about that, but it would also be great to hear what you don’t do.

Sean: I think the thing that you don’t do — and I’ll make one caveat in a second, on this — is that you don’t aggressively try to grow the business. Particularly, you’re not trying to do business development relationships. Maybe you can spend enough to create some flow so you actually get people giving you the feedback on the product, but one thing I wouldn’t do is obsess over ROI on that flow.

Your goal is not to figure out that repeatable, scalable customer acquisition engine at that point. Your goal is to get enough people in there to where you can react to their feedback and hone the product experience on their feedback.

Should I launch?

Nivi: How about a launch?

Sean: This is very debatable. A lot of people would say that a launch is a really important event. For me, I’ve never been big into launches. Launch is a one-off. With launch, you’re going to get a group of people in one time, and you’re going to get some feedback on them. And yes, they may be cheaper because you got some good press around it, but you haven’t learned anything about how to grow the business with a launch. So for me, I’m much more….

I wrote a blog post way back when. I think it was called “Launch with a Trickle,” and that’s much more of my focus on things. Just get enough people on there as early as you can, so that you can react to their feedback. And you may find that they absolutely love the product. Then hone in on why they love the product. And then get the messaging right and the experience right, so that you’re delivering that better. Once you’ve kind of tightened up and you’ve got that validation that a lot of people love your product — hopefully more than 40% say they’d be very disappointed without it — then you can step on the accelerator and a big launch might make sense at that point.

You can’t set a deadline on fit

Sean: But what I see too many companies do is set a date. They say, we’re going to have our launch on January 14th, and they’re two months away from that and they’re working towards… I mean, it’s either time or it’s not time.

Nivi: Yeah, and you can’t set a deadline on product/market fit. Right? That is the point in time, in the growth of a company, where you basically are unable to predict anything.

Sean: Right. And all you’re going to do then is make the mistake of being too aggressive too early, or piss off your investors by missing that date for the launch.

So it’s much better to say, our clear objective — the next milestone that we need to get to before we do all of these things — is to actually make sure that enough people love this product. As soon as we get there, then we have to figure out why, and make sure that we’re really honing every piece of the business to reflect why people love the product.

How to communicate with the board during fit

Nivi: Right. So, on that topic, how do you communicate with the board and set expectations when you’re trying to get to the product/market fit stage?

Sean: What I’ve found is that people either get it or they don’t.

Nivi: What are the responses of the people that don’t get it, like from a board perspective? What have you heard board members say directly, or secondhand?

Sean: They’re just: You’ve got to go for it! You’ve got to go for it! It’s good enough. It’s good enough. And they just want to accelerate the business.

I have a great example of a company that was about 15% or 20% on product/market fit and I loosely advised them at no cost, because I was going to spend some time in Scotland and they’re based in Scotland. And I wanted to get to know these guys better because I was going to be spending a week over there. So we just worked together over a period of time.

And when we first started working together they were at the 15% or 20% very-disappointed number, and they explained to me that their board of directors was just pushing them on: We’ve got to get to this growth number! We’ve got to get to this growth number!

And they took the time to really explain: This is our goal right now. We don’t think our product is good enough to accelerate this business. Our goal is to take the feedback from the users that we have and keep getting our product better.

In the process of doing that, they had the very painful process of actually having to lower projections on growth, and doing it week after week, and frustrating their board of directors through that process. But they did exactly the right thing, because today their very-disappointed number is over 50%.

And when I met with them when I was in Scotland about a month-and-a-half ago, they were at a really exciting point where they said they keep going back to their board of directors and saying: Hey! We’re going to revise those numbers up. They kept going and saying the numbers are too low. It’s going to be better than that, because now that they have the product right they could really focus on that, and they weren’t going to have the….

You can grow when your product/market fit is lower than that, but you’re going to grow a lot slower than you otherwise would, and you’re not going to address the real problem. And you’re going to have a very frustrating experience trying to grow that business over the next several years, versus saying: Let’s keep our costs really low, let’s get the product right, and when we’re focused on growth, no excuses. Now we’re absolutely accelerating that business and it’s going to be much easier to grow the business at that point.

And ultimately, you take it a year out or two years out, you’re going to have a much bigger userbase doing it that way than if you’re trying to grow through the whole period of time.

How to use positioning to improve fit

Nivi: Now, what changes should they make to get to that number, without getting into the specifics of the business?

Sean: I don’t know. I didn’t work with them, so all I know is that they did the things that I’ve mentioned, where they really took the feedback of the very-disappointed people, really tried to look for the signal of who loves this product. Somebody loves it. Who loves it? Why do they love it? How can we really put a stake in the ground around that and try to make that piece better, and try to play down the other pieces?

Nivi: Then maybe reposition?

Sean: Yes. Some of it’s definitely through repositioning.

I have another company that I worked with that was before I had [done] that. This was really the company that helped me come up with the idea that I was going to survey people ahead of time, because I got into two companies at the exact same time who were both at about [where] 7% of their users said they’d be very disappointed, and found myself in the “oh, crap!” moment of figuring out that now I’m going to have to help these guys transition to growth and grow, but they’re not ready for it yet. And why did I wait until I had a six-month contract with them to run the survey? From now on I’m going to survey it ahead of time.

But at one of those two companies, we were able to see a really strong signal in the 7% that said they’d be very disappointed without it, reposition on that signal… [interrupted]

Nivi: So how do you reposition on the signal. In this case, did you get a little bit more data…?

Sean: We just found out — I mean, I can’t give any specifics on it — they had a product that had lots of different features, and each one of those products could be a product in itself. They built a really big suite of tools, within a product, and everyone who said they’d be very disappointed was focused on just one of those. So we basically only talked about the one they were focused on, and we didn’t talk about any of the other things.

Nivi: To the user, say, on a landing page or on the home page.

Sean: Exactly — on the home page we only highlighted that and said we’ll introduce the other stuff after we get them in. One of the benefits with that was that we were really able to simplify messaging. Having a really complex product set that you have to present people is way harder than simplifying that and only highlighting one. So, we basically highlighted the most important must-have product from the group, and hid some of the other products, and as a result we could be much clearer in what the benefit was for the people coming through and what we were actually offering.

So, one week later, on the next cohort of people coming through, we were over 40% because they had a very different expectation of what this product was, just by changing the messaging and focusing their first user experience on just this one piece of the business.

Nivi: Right. So, with no real change in the product, but a change in the messaging and a little bit of first-user experience change, you got a huge uptick in the must-haves.

Sean: Yup, exactly. And the interesting thing is that business, probably out of every business that I’ve worked with, is the one that I think has the opportunity to be a multi-billion dollar business, and one that I’m most excited about. And when I first saw that 7% number, it was the one that I thought: how do I get out of this? So now I’m really happy that I stuck with it, and I’m happy that we were able to figure those things out. And I’m really good friends with the founders.

It’s frustrating to try to grow without fit

Sean: When everything’s going well, the chemistry gets much stronger and it’s just a much more enjoyable place to be. That’s just one of the things that I’ve learned, is that in a company with a low product/market fit, it’s not a fun place to be. It’s frustrating trying to grow that business.

And for a company that can really embrace the fact that: You know what? We missed. The product that we created just isn’t something that people really seem to want or need.

Nivi: And we don’t really measure ourselves through growth right now, either.

Sean: Yeah. So for us, we’ve got a lot of money in the bank. Somehow we got lucky enough to raise the venture capital. If we try to grow we’re going to waste a lot of that money trying to grow. Let’s just keep our costs way down, get just enough users in there and either do a complete reset on the business — like PayPal did, which I think most companies don’t need to do — or just find the love.

In my last blog post I talk about “just find the love.” Somebody loves it. Somebody really needs that, and once you find that, that’s the stake in the ground that you can figure out how to create a successful business.

How many users do you need to determine fit?

Nivi: And what is just enough users?

Sean: That’s a good question. I think it kind of depends on the business, but ideally, if you have a product that ultimately is going to cost thousands of dollars, then it’s just not going to be realistic that you can bring in a hundred every day on that product. But if you have a product that’s going to have a 10% upgrade rate or a 10% purchase rate, and it’s going to be $100, then that’s a product that you probably want to try to have a pretty constant flow of around 100 new people a day.

And whatever it costs to get to that 100 new people a day is a really good flow to iterate on. If you can only be at 30 or 40, that’s probably OK.

Nivi: And how about on the output side of that — the people that you’re going to survey, the people that have gotten to the finish line? How many people do you need to be talking to, to get a good signal there?

Sean: I look for around 30 responses as a minimum. It really depends. If you have a very low percentage of users that say they’d be very disappointed… Say you’ve got 30 responses and you’ve only got 5% that say they’d be very disappointed without it, that’s not a whole lot to work with on finding a signal.

But I ran a survey this morning with a business that has 70% of their users that say they’d be very disappointed. We got a very strong signal on 42 responses.

Nivi: OK. Right.

Sean: Really, it kind of depends, first, on how strong that number is. And then what you’re trying to do is get enough responses in there to find some sort of signal that either says: We completely missed, or that [there are] very few people we’ve hit with, so we want to make sure that we survey enough people that we’ve got maybe 10 or 20 people who say they’d be very disappointed, that we can start to hone in on the feedback that they’re giving.

Who do you survey?

Nivi: And do you ever have trouble defining what that finish line should be? For example: active monthly users, people who just signed up for the product, or is it… [interrupted]

Sean: Do you mean on who to survey?

Nivi: Yeah.

Sean: What I tend to do is say, people who have been back at least once.

And I think for a consumer product you shouldn’t be charging at this point; it should be free at this point, because the money side already starts to put a big filter out there. Even if you had a trial, people would make up their mind pretty quickly. If they decide they’re not going to buy it, then they’re not going to say they’d be very disappointed without it, and not buy it. So, just pulling money off until you can get that signal is pretty important.

For me, I define what is actually the experience that we think is the experience that gives them a full taste for the product, then I want to make sure that people have at least done that when they come back to us.

Nivi: So in the case of PayPal what would that be?

Sean: That they’ve at least made a payment with PayPal or accepted a payment.

Nivi: One payment?

Sean: Yeah. And then, that they’ve been on the site within the last two weeks.

Nivi: OK.

What is promise?

Sean: So those would be kind of the two pieces. You just asked about intent and the value proposition in some of those things, so I think the important starting point is that the promise or the differentiated value proposition is really based on whatever signal that you find. Where you find the love on the product, that’s what you’re trying to do there.

Nivi: And you call that promise.

Sean: Promise is one of the names I use for it, but it’s all synonymous with just the value proposition, the differentiated value proposition of just what is this product going to do for you. What is the unique piece about it?

Nivi: Right. So in the PayPal case: send money to friends online.

Sean: Yeah.

Nivi: OK.

Sean: Yeah. I think that would be… And at the time, that was fairly differentiated. Now you can use credit cards and other things, so something that remembers your log-in information might be trying to differentiate from just Visa or Master Card or American Express.

Don’t over-position your product during fit

Sean: But what I was going to say with that is that there are two periods where that becomes important. One, before you have product/market fit you’re trying to find that signal, so you can do some sort of early positioning to try to highlight that. But the problem is, too much positioning is going to steer your users in a certain direction. So, early on, a lot of times it’s good to actually be very feature-function oriented.

Nivi: OK.

Sean: Because unless you have a real strong signal…. You can experiment with it, like, OK, let’s bring people in and we’ll give them this message and we’ll see what their very-disappointed number does, just on this group of people. We’ll survey them separately.

But in general, if you don’t have a real strong signal, then it can be kind of dangerous to shape their experience by highlighting some unique attributes that turn out not to be the important unique attributes.

Nivi: Because it might be inconsistent with…?

Sean: Because you’re going to shape their experience so much that they may miss the other thing that would have been the thing that would really thrill them.

Nivi: Right. So give me an example of how I would do that.

Sean: X Corp. is a prime example on that. In the early days at X Corp., it was just one of those things that I realized that if I really focused on homepage messaging it was going to take me a really long time to get some level of consensus, and that’s just the way that the business….

The way most startups operate, early on, is that everyone has opinion around what the homepage looks like and what it says. And so, what I realized is that getting a lot of those early users, we were going to be getting them through search anyway, so I basically said I’d let everyone else figure out what’s going to be on the homepage, I’m just going to take people in through landing pages.

With landing pages you know what the person’s intent is when they’re coming in through a specific search word, so you can really hone things in a more segmented way.

But our homepage ended up being very feature-function oriented, and the group we initially built the product for ended up being a relatively small percentage — especially on a revenue side — of where the business ultimately ended up generating a lot of revenue. So I think if I had taken a traditional marketing approach to that I would have been very benefit-oriented right from the beginning. And I probably would have positioned it away from what ended up being the really interesting market for us, because I would have given so many details about how they should use the product that this other group may not even have considered using the product.

And so, as a result of having, essentially a group of engineers be able to initially determine the wording for the homepage, they just focused on the features that they created, with not a lot of why someone would need it. And that turned out to be the best thing that could happen because, ultimately for us, by the time I really started looking for that signal, the signal was very different than what we would have initially had if I had positioned out of the gate for that.

Pivot the business around the love

Sean: A big theme that I keep coming back to is, where is the love? It’s basically just finding out which users are passionate about the product. What is the use case on the product that those people have? Why do they actually love the product? And that gives you a good core of information to guide every part of your business. So messaging, you want to reflect that, but also, now that you know that, you want to make sure that you can really get a lot more people to that particular type of use case and that particular type of gratifying experience. So understanding user gratification is really critical in all of it.

First of all, you want to know that you have enough that you can try to grow the business — that you actually have something that will be growable. And then in growing the business, having that gratification should give you guidance on everything. So product roadmap should be a function of how people are gratified with the product and who’s gratified, so it should give you a lot more focus on that product roadmap as opposed to a lot of feature feedback and ideas and different things that, over time…

You know, initially you have to kind of scatter-shoot it. You have to cast a fairly wide net of things that might please people, but once you really know where that gratification is, that’s where you really want to double down, and that’s your business. Before, everything’s an experiment. But your business is really on where that gratification or where that love is.

Nivi: Does that go to what is the primary benefit of the product in your Survey.io?

Sean: Yes. That’s where I start to see it. I start to see it in each of those questions. I can start to see, in the very-disappointed question there’s a: why did they pick that? You can start to see it in there, as well.

Ultimately, that’s the first thing that I’m trying to look at. One, do enough people love this product? And two, why do they love this product? And then everything that I’m doing is basically helping the business to — I wouldn’t say reposition, because reposition is associated with messaging — realign itself around that really strong user gratification. And once you have that alignment around that, it gives you a lot more direction.

Early on, everything is an assumption, everything is vision, but once you have users on there you can finally start to find that nugget of value.

Focus the product on the love

Sean: And one of the things, if you really step back and ask a lot of entrepreneurs why startups fail, a lot of people rightly focus on lack of focus as being what causes a lot of startups to fail. But then the question is, OK, if it’s lack of focus, what should you be focused on? Obviously, lots of focus on the wrong thing is still going to cause you to fail, so then it’s a big question of what you should be focused on. And for me, ultimately it’s that strong signal that this is something that users really love about the product. So understanding who loves that product — who really has that need — that starts to give you guidance on who you’re going to target going forward.

What other things can we do that can enhance that use case — that can make that particular use case? How can we prioritize those feature enhancements, the parts of the product roadmap that relate to that, and maybe drop some of the things that are the wild cards that we’ve had in there for a while?

The thing that amazes me is a lot of startups have got a little side-product project of a completely unrelated product or something else, which I think, early on when you don’t have product/market fit, is probably not a bad idea. You’re throwing a lot of stuff out there to start to see what sticks.

But once you’ve found that signal, you’re ahead of 90% of the startups out there. So once you find that strong signal, your ability to be really successful is focusing on that strong user gratification and passion and figuring out how to get more users to it, more of the right types of users to it. How can we make it even better — but, that part better, not all parts better, that part better?

Create a great experience around the love

Sean: How can we completely refine that brand experience?

Now is the time to go and get a better designer to come in and really get the graphics perfect on that part. Now is the time to bring in someone who is the absolute wordsmith to get the descriptions perfect on the website, but making sure that they’re being very responsive to that user feedback and they’re not just being a wordsmith by what they think is what’s important to people.

Create a business model around the love

Nivi: Business model?

Sean: Business model is absolutely a function of that. You want to make sure that you are not trying to monetize something that’s not important to users. If users tell you they’d be very disappointed without it, the other thing that they’re telling you is that you could keep cranking that price up for a long time before they change to being somewhat disappointed without it.

And if you’re really focusing on that to determine your price and how you charge for the product, you’re going to have a lot more signals.

First find the love

Sean: So I think that’s a big theme that startups should look at, is focus, and focus on what and when.

The “when” — early on your focus is on just trying to find some love within your product. Find somebody who loves something about your product. And you can get that day one, before your product is even developed, just by talking to people and saying: Hey, this is the problem that we think people have out there and this is how we’re going to try and solve that. Does that seem like something that would be useful for you?

Nivi: The customer development approach.

Sean: Yeah, the customer development approach. Steve Blank has a really good book that’s documenting everything on that and has written a lot more about that.

I purposely, again, have stayed out of that area because it’s so high risk, and you can’t be good at everything so that’s the part that I’ve said I’m not going to try to be good at.

But essentially from day one, it’s all about that search for something that somebody really needs.

Are recommendations a good indicator of fit?

Nivi: Going back to the must-have — that metric — you don’t see user recommendation in that metric there, as well?

Sean: It’s interesting; I see a lot of correlation on success in businesses with the must-have.

I had somebody contact me recently who had a 70% must-have number and they are adding 20,000 new users a day with great monetization, and they just started recently. It was just one of those things that wasn’t surprising to me to see all that success because they have that strong must-have number without really even having a marketer in the company.

So the must-have, I think, is really big, but there are a lot of people that have good recommendation numbers that struggle, and part of that is that… I think YouTube videos is a prime example that sometimes you just see some really funny things that you want to share with other people, but ultimately would you be very disappointed if you couldn’t? Probably not.

Nivi: Right. So it could be a false-positive. But let’s say for the folks that do have the 40% must-have, do you see a correlation of recommendations then?

Sean: Not necessarily. Sometimes I see it really high. A company that I surveyed today, with surveys rolling in right now, was 90% recommend rate, but sometimes I see it being pretty low on a company that’s doing really well. Sometimes I see, as I said before, companies that are pretty low on the must-have, but high on the recommendation rate. So I actually don’t think that’s a good proxy for success, but I haven’t studied it enough to know for sure.

The must-have metric is a good indicator of fit

The only thing that I do know is, on the must-have number, I’ve probably looked at 150 companies now on that number and I could maybe point to one or two that were less than 40% that had any kind of traction and success. And over 40%, everybody had some success. And maybe it’s 35%, maybe it’s 45%, but somewhere around there you get enough people.

The only thing I tend to find a challenge with is that some companies have a must-have number that’s around 40%, but then the feedback is all over the board on why they consider it a must-have and that becomes really hard for being able to find any kind of signal to base the business on. So that’s a different problem, and I haven’t figured out how to deal with that.

Nivi: There are too many segments in the customer base.

Sean: Yeah. Everybody loves it, but for different reasons. Then maybe you have to figure out which one of those reasons represents the biggest target user group and hone in on that.

What comes after fit?

Nivi: Right. One thing you talk about is, once you have the product/market fit, trying to get through the next few steps as quickly as possible. So it would be great if you could talk about that, and also, for people that don’t know what the pyramid looks like, what are the next few steps?

Sean: Sure. As I said, if you don’t have product/market fit, you want to be obsessively focused on getting there, however you’re defining product/market fit. I think the easiest kind of definition to work toward is trying to get 40% of your users to say they’d be very disappointed without it.

And during that period you’re trying to stay very low burn and very conservative on all of your expenses within the business. And once you find 40% of your users that say they’d be very disappointed without the product, then you’re in a position that you have a business that can grow now. So then the question is: do you try to grow the business right then or are there some things that you can still do that are going to make it even easier to grow when you’re focused on that? And what I’ve found in working with so many companies at this stage is that you’re definitely better off waiting a little time until you try to grow the business.

Nivi: And what does it mean to grow the business? You’re talking about spending money to acquire users, essentially.

Sean: Acquiring users, for a lot of businesses, means starting to spend money, but it might also mean really trying to crank up the virality of the business, or it might be SEO that doesn’t require a lot of spending.

Nivi: But it’s time and money that you require for people… [interrupted]

Sean: Yeah, and focus. The time piece, you could say, has time or focus. But it’s essentially saying…

Music: Tortoise

Is two co-founders the only way to go? How do we split up the company? Who’s the boss? How do I know when to compromise on a co-founder?

So Naval and I recorded a 40-minute interview to answer your questions, explain how to pick co-founders in detail, and describe not only what to do, but how and why to do it.

The interview comes in two versions: Mini and Pro.

Mini

The Mini version of the interview is free. It includes the first 5 questions of the interview — audio and transcript. Voila:

Audio: Mini interview with chapters (for iPod, iPhone, iTunes)
Audio: Mini interview without chapters (MP3, works anywhere)
Transcript: How to pick a co-founder (Mini) (PDF)

Pro

The Pro version is $9. It includes the full interview, with all 16 questions plus:

• A nicely-formatted 48-page transcript. Here’s a sample.
• An MP3 you can download for your portable device or media player. Here’s a sample.
• Chapters, so you can jump to the part of the interview you want. This only works on iPods, iPhones, and iTunes. Here’s a sample.
• Your money back if you don’t like the interview.
• THAT AWESOME FEELING OF BEING A PRO.

Get the interview for $9. One-click checkout & download. #

“Great interview. Well worth the $9.”

– Richard Burton, Hoodeasy

Or (!) buy our Happy Meal: This interview, our Cap Table, and our Pitching Hacks PDF for $19

30% cheaper than buying them one-by-one.

Not interested? That’s cool. Tell us why and we’ll send you another one of our products for free.

Questions

Here are the questions we cover in the interview:

1. How many co-founders?
2. How do you create a history together?
3. How should you divide up the company?
4. Who’s the boss?
5. Do you even need a CEO?
6. What skills do you need on the founding team?
7. Why do you need aligned motivations?
8. Should I compromise on a co-founder?
9. Why should I partner with a nice guy?
10. How can I tell if the other guy is a good builder/seller?
11. How do I convince someone to partner with me?
12. What if my co-founder needs a salary?
13. Where do I find a co-founder?
14. How do I start a business with family?
15. How do I start a business with friends?
16. How many co-founders? (Redux)

And here’s a transcript of the first 5 questions (also available in PDF).

1. How many co-founders?

Nivi: This is Nivi from Venture Hacks.

Naval Ravikant: And Naval from Venture Hacks.

Nivi: We recently posted an article by Naval called “How to pick a co-founder.” You probably checked it out. If you haven’t, take a look.

We got a lot of questions based on the article, and I also have questions based on the article, so we’re going to answer your questions. We’re going to try to elaborate on what we wrote and try to give some more examples of some of the things we talked about.

So, the most common question we had was about two-founder companies. Is two founders really the only way to go? What are the pros and cons of two founders and other types of situations?

Here’s some of the feedback we got:

Ram said, “As you point out, there are several advantages to having two co-founders, but I think your perception exaggerates the odds against single-founder companies. From this week’s news, AdMobs is a great example of a successful single-founder company.”

Gabor says, “Was Facebook really a one-founder company? Maybe I’m bad at my Facebook founding history, but it seems like there were multiple. It was one-founder in the sense that Mark Zuckerberg is both a builder and a seller, but I feel like there have to be more good examples of successful one, three, and four-founder companies. It seems like you guys dismiss anything other than two, too quickly.”

And then finally, Mark Essel says, “I don’t see three to four being a problem with the right group. Certain projects lend themselves to two; others can really get powered up by three to four founders. Engineer-heavy startups are common now, so doubling or tripling your development team at the outset is incredible. It depends on what other responsibilities team members have as well. Many startups are grown while folks work their day jobs, or part time to survive.”

So, the first question is, are we dismissing anything but two-founder companies too quickly?

Naval: I would say that’s not the case. Of course three-founder and one-founder companies can be successful, it’s just that generally speaking, over mass statistics, if you look at many, many companies, especially the bigger ones, it’s usually two founders that really matter.

Now, you may start with three founders or four founders, but very often the company tends to coalesce around two founders.

Or, you may start with just one founder, but that founder may have a very early and senior confidant who then ends up almost playing a founder role even though they join the company later. For example, if you look at Microsoft, Paul Allen stepped aside after a certain point in time, and Steve Ballmer really became the guy who came in and was Bill Gates’ co-founder.

So, I guess the important things to point out are that it is very, very important to have a strong, day-to-day partnership element when you are going through something as difficult as doing a startup. And whether that’s with a co-founder or with a very early person who joins the company and fulfills that role, the titles matter less.

The issue with three or four-founder companies is not to say they can’t succeed, it’s just that it is much, much harder to divide up the roles in such a way that there isn’t stepping on people’s feet. It’s much harder to find people of roughly equal caliber.

And then as a company progresses, in the much later stage, especially for a venture-backed company, you just end up with a lot less equity to go around. And you don’t want to be in a situation where 10 years down the road – and by the way, it takes 10 years to build a truly great, huge primary franchise – after your company’s gone public, you’ve raised lots of venture capital, you may find that all of the founders, because you had so many of them, own one, two or three percent of the company, each, and there’s nobody, really, who can take charge of the company and be the CEO and just drive it through thick and thin.

That persistence, that drive that’s necessary in a founder, is important to the health and success of a company, not only early on, but also much later in the game. And as a company gets older and older, it becomes less likely that a four or five person company can sustain the equity stakes required.

Lastly, I would say that any time you have three people in a room together working on something, you get politics. Then it starts mattering who says it, how they position it and whether they can get two to gang up on the third.

One of the nice things about a two-founder company is both have to agree – at least in a well-structured two-founder company.

Nivi: OK. You went into a bit of the pros and cons, so why don’t we talk a little bit more about the pros and cons. Three to four founders: what are the pros and cons? Let’s catalogue them.

Naval: Well, the pros are, obviously, many hands make light work. So, if you have three or four highly incented founders, early on, and they’re motivated like founders, they’re just going to accomplish a whole lot. That’s probably the single biggest pro.

You also have a diversity of voices and opinions, which can work both ways. It can be great when you’re brainstorming and problem solving; it can be really bad when you’re trying to make decisions.

One of the early problems a three or four-founder company struggles with is who’s on the board. Because a lot of control and authority derives from the board, every founder wants to be on the board.

And VCs don’t like that. They know it’s dysfunctional when you have four founders on the board because then they can’t really talk about each other or who’s doing a good job and who’s not.

The CEO also ends up having very diluted authority, because now he not only has to just command the respect of one person, he has to play a little bit of politics and keep everybody happy. So it ends up being a difficult management situation.

Nivi: How about pros and cons of two people?

Naval: The pros are, obviously, that you just have to establish one solid partner relationship, there’s a clean division of roles and responsibilities, and there are minimal politics.

The cons are, if those two can’t get along the company is effectively dead. You have no room for error or failure. If the people don’t get along you’re going to have massive clashing. You’re going to have too much DNA and time burned up internally.

And you just don’t have as many people to go around, so the two have to be extremely good at what they do, which is kind of why I made the “one builds and one sells” distinction, because although you can occasionally find the superstar who can build and sell, selling is a full-time job, building is a full-time job, so you might as well just pick the best on the planet at each.

Nivi: Right. And how about the one-founder case pros and cons?

Naval: I think it’s an extremely difficult case. I would recommend the three or four-founder case over the one-founder case.

Nivi: Right.

Naval: Well, let’s start with the pros. The pro is that you get to keep a lot of equity. [laughs]

The con is that you have to do all the work.

Another pro is that it is completely your vision. It’s one person’s monomaniacal vision, which sometimes can be great. I don’t know the histories of these companies, necessarily, but it sure feels like Salesforce.com is a one-founder company, with Marc Benioff having the bully pulpit.

So, it can definitely be one person’s vision and one person’s drive, and so forth, but that person had better be really, really driven and really confident, because every startup goes through tough times, and you won’t have a shoulder to cry on, not really. You won’t have someone to pull you up.

You also won’t have a diversity of opinions so you’d better be right and just able to sustain it. But now you’ve basically got a company that is just an extension of one person’s personality, with all the ups and downs of that. I think the proper analogy might be trying to raise kids by yourself; sure it’s possible, but it’s a lot easier with two.

Nivi: I would say, whether you have a two-person, three-person, or n-person team, where n is greater than one, if you have a functioning team, don’t worry about everything that we’ve said too much because a functioning team – an effective, well-functioning team – is basically the rarest thing in the world of startups. Markets can be put into place. Right? You can pick a product, you can pick a market, but it’s hard to pick a team.

Naval: Absolutely. The first company I started had three founders. The second one had five. So, if it works, it works; if it doesn’t, it doesn’t. You should not go and remove founders just because you’re over two.

But, I would say if you’re a one-founder company and you’re finding things to be just difficult and every day is a chore, you may want to consider bringing on, even at a later stage, an early employee or late founder – someone who can be your partner. It’s that partnership element that’s very important.

And keep in mind; human beings are pair-bonding creatures. We’re evolved that way and so we’re just designed to operate that way, and operating on your own for long periods of time in sustained difficult efforts is just highly unnatural.

2. How do you create a history together?

Nivi: You suggest people find someone that they have a history with – you wouldn’t marry someone you just met – and that you should date first. And you suggest going through something difficult, like a prisoner’s dilemma or a zero-sum game. How would you actually do that in real life?

Naval: Basically, what you want to look for are people who are cooperators who you can get along with. So, when I say prisoner’s dilemma or zero-sum game, that just means go through a tough situation with them. A tough situation can be one in which they can gain at your expense, and that’s really how you know someone. So, if you’re dividing up some money or if you’re trying to figure out who should do the dirty job or the hard work, you want to be with someone who’s going to volunteer for that kind of stuff, who’s going to volunteer to do a dirty job.

Steven Levitt and Stephen Dubner, who wrote Freakonomics, wrote a sequel to it called SuperFreakonomics that just came out, and in the prologue they have a very interesting and kind of funny little quip where they say: We were trying to figure out how to divide up this enterprise and we both wanted 60/40. At first it didn’t feel right, and then when we realized that each of us was offering to give 60 to the other guy, and wanted 40 for himself, that’s how we knew it was the right partnership.

So, I think you want to try and find that spirit of cooperation, because you will go through tough times, so you need to have some way to predict how the other person is going to behave in those tough times.

Nivi: So, one practical way to actually play this game would be to lowball what you think should be your equity percentage in the business and see how they react to that.

Naval: That’s actually a really good test. Most people would fail that test, which is a good screen.

Nivi: Right.

Naval: But when your partner, in quotes, turns to you and says, yeah, that’s great, I’ll take 55, you take 45, or, I’ll take 60, you take 40; now are you really going to get up and walk away?

Which is a natural segue into: how should you divide up a company amongst founders?

Nivi: Cool. Let’s talk about that.

3. How should you divide up the company?

Naval: I’ve seen lots and lots of different examples and cases, and at the end of the day what I found is that, again, there are multiple ways to do it, but far and away the most stable configuration is one in which it’s a 50/50 split. It’s an inherent fairness.

Yes, not everybody is created equal, but it’s very hard to measure when people’s contributions are important. Some people’s contributions are very important early on. Some people’s contributions are important later on. It’s very hard to measure how much effort someone is putting in. It’s very hard to measure which crucial deal made the difference. So, I think a good rule of thumb is, start working at the same time, both work at the same salary or neither salary, and just make it 50/50.

I’ve seen cases where it’s 55/45 or 53/47 or some unnatural number made up, and the bad blood surfaces five years down the road when you least expect it.

Nivi: And what about the case where you’re basically hiring the first employee, perhaps calling him a co-founder, and giving him a large equity chunk – say 10% – but it’s not a 50/50 split. Is that common or is that rare? What do you think about that?

Naval: It’s not common, but it’s not uncommon either. I think it’s a pretty good way to go if you don’t have a co-founder, or even if you do have a co-founder but it’s someone who joins you early on and they just do a stupendous job – they behave like a founder.

And by behaving like a founder I mean they take responsibility for the outcome of the company. They work above and beyond the call of duty repeatedly. They stick with you through thick and thin. They voluntarily sacrifice their salary or even things that might seem like it’s good for them, for the betterment of the company. And if you see someone who’s doing that and becomes indispensable to the company, I’m a big fan of giving them a lot of stock and treating them like a late co-founder.

Now, later in the game you can’t go handing out huge chunks of the company without getting in trouble with employees and investors, but you should do the best that you can because companies take five or ten years to build. They’re just the sum of the people who are involved, and if you have a superstar person early on, you need to recognize that before you lose them.

4. Who’s the boss?

Nivi: So, let’s say I’ve got two co-founders with a 50/50 split. Who’s the boss? How do I figure that out? Is there going to be a CEO? And say we have one board seat. So, we raise some cash from a small, early-stage VC fund. There’s one investor on the board, one founder and one independent, or just one VC and one founder. Who’s on the board? Who’s the CEO? And who decides when a founder can be terminated?

Naval: Traditionally, this is a very difficult question. This is why it’s very important to have a tremendous trust-based relationship with each other. You have to trust that even when you’re not in the room, the other founder will take care of your interests as he or she would take care of their own.

If you don’t have that level of trust with someone, then you cannot answer this question. Assuming you have it, it should almost be easy.

People should be saying: “No, no, you take it! No, no, you take it; I don’t want to deal with it.”

The truth is that CEO in a startup is a tough job. It’s not a fun job. You deal with a lot of the crud. You have to clean up a lot of the employee issues, personality disputes, keep the investors up to date. It’s actually among the less fun work. Anyone who thinks that CEO is a really fun and sexy job probably shouldn’t have it. I would think that with two good founders who trust each other there should not be much of a dispute.

Ultimately the board is the arbiter of control in the company, and the board is elected by the shareholders. It’s quite common for there to be a voting agreement that forces drag-along. I think it’s pretty important that if you want to have a say in your company and control in your company, then you reserve your right to vote as a board member.

So, in the two-founder case, you should not sign any voting agreement that allows just the other founder to select a board member or names and fixes a board member forever. It should be by mutual consent and you should be able to bring it up for re-election, and that will force there to be some level of good behavior and an alignment of interests.

In terms of who fires who or how does a founder get fired, you only want a founder to be able to get fired if – a) the other founders agree, and b) you also have some neutral, outside independent arbiter, which could be a VC or independent board member.

5. Do you even need a CEO?

Nivi: Here’s a question. You’ve got a team of two guys who started a company, and maybe you’ve hired a few employees and you’ve got about five people. Do you even really need a CEO beyond the guy who is on the board who is somewhat on the hook to the investors, if there are investors on the board?

Naval: I think you need a CEO in the sense that, on a day-to-day tactical level there has to be someone who’s just making decisions so that not every decision gets caught in a situation of he said, she said; or you don’t have people going to one person, and if they don’t like the decision they go to another person and lobby. That’s a very inefficient process.

But at a strategic level, month-to-month you don’t need a CEO in that nothing is going to happen that is huge and material for the business without both partners agreeing. So, in a five percent company, a CEO is not a guy who goes and cuts a deal or raises money without consulting his partner, but is a guy who decides on a day-to-day basis, yeah, we are going to go ahead and make that slight tweak to the side, or, yeah, tell a lawyer that that interpretation of the options contract is OK, and so forth.

Nivi: And then, in terms of getting terminated, at the very early stages where it’s me and another co-founder and maybe a few employees, I basically don’t want to be able to be terminated without the consent of the board. This should be a board-style decision. Like, the CEO, whoever that is who has that title, can’t get rid of me while we’re in less than five employees or a few employees, right?

Naval: Yeah. Like I said, you want to have a process in place, and that process should require the consent of both the other founders and some independent arbiter. It’s pretty hard to fire a founder, and it should be hard to fire a founder.

That being said, all founders should be vesting because the worst-case scenario is that if you have to let go of a founder and that person owns 30% or 40% of the company, there’s no vesting so they walk off with all of it. Now you have very, very early-stage company with only 60% or 70% to go around and it’s just not enough to build a company around.

Intro/Outro Music: Entertainment Tonight

At IMVU, Eric and the management opened up board meetings to the entire company. Why?

1. To give people the information they need to do their jobs.
2. To teach everyone in the company to think like the CEO.
3. To prevent employees from gossiping about board meetings.

And more!

I’ve synchronized the audio with some simple slides below. That’s my favorite way to consume the audio. You can also find a transcript and stand-alone audio below. Please let me know if you find the transcript useful.

Read on to learn what kind of employee Eric used to “show the door” at IMVU…

Slides: Opening up board meetings (pdf)

Audio: Opening up board meetings (mp3)

How do people find out about our product if we haven’t launched?

Nivi: And this gets to a second topic, which is you guys weren’t doing any of this really in public, because you had not launched a product, right?

Eric: Amen.

Nivi: Nobody knew who you were, but people were, at the same time, were using the product, so how did you do that?

Eric: I got a question today which was something like, “I’d love to follow your advice about not having a public launch, but we need to get early beta users for our product launch. How can we do that if we are not willing to talk to bloggers? Nobody really knows who we are.”

I think a lot of people have that attitude, that without PR, you just can’t get any early customers. Again, we have got to start with, “What is the goal of early customers? Why do you want them?”

If you are charging from day one, one of the reasons you want them is you actually want to make money. You want to show that your business is viable. But even if you are not charging money, you have a need to find out whether your business is viable, whether you have that minimum viable product, whether the business model, at the end of the day, is going to work.

For that, you do need customers, and you do need to be putting customers through a product experience that will give you that information. But you don’t need a lot of customers. I think that is where people get confused.

For a big fancy launch you can get hundreds of thousands of customers to show up for one day. But for metrics analysis, generally a cohort of 100 people or 1,000 people are plenty to learn from.

If you change your goal from, “How do we get the maximum number of customers,” to, “How do we get the minimally sufficient number of customers to learn what we need to learn,” new possibilities get opened up to you.

Acquiring customers on $5 a day

Eric: For example, at IMVU, we practice the $5 a day AdWords campaign. I was the VP of marketing in those days. If I actually knew anything about marketing, I would have known not to try this. By traditional marketing standards it is considered crazy to spend only $5 a day, but we had a pretty low budget and we really were pretty frugal.

I discovered that in those days you could buy clicks for five cents a click. But to me, $5 a day meant every single day 100 human beings are coming to try my product.

If you think about that from a beta testing point of view, especially if you look back at the old days of software shipped by CD, getting 100 people to try your product is actually a lot and you can learn a lot from that. And at 100 people a day, you are in good shape, just at that tiny, tiny level.

The risks of doing that are really quite low. I think a lot of engineers have this idea that once you put your product out there in public, the investigative journalists are going to find out about it and write about it and we are going to lose control of the story. Let me tell you. You should be so lucky.

IMVU was a top 1,000 website in the whole world before it got any press whatsoever. We were making millions of dollars a year. The press was writing about newly funded, venture backed competitors that had no traction whatsoever; because those were the guys sending them press releases.

It was frustrating, and psychologically you want to have that cover story on WIRED that you can send home to mom, but you know what? We did not start this company to have good vanity covers printed about you in the press. We were there to serve customers and serve them well.

Running experiments under a different brand name

Nivi: How do I run experiments, if I already accidentally got that TechCrunch article and I…?

Eric: Yeah, I am sorry. You are not doomed, but you are going to have to go waste energy later cleaning up the positioning that you put in that article, which is undoubtedly wrong.

Nivi: Right. There is that aspect of it, but do you think you should, just basically pretend that article never existed, or do you run tests under a different brand name?

Eric: That is not a bad idea. Especially on the iPhone, I see this because of Apple’s stringent release process where there is this huge delay before you can actually bring things to market.

And also because people want to get into the top 25. That is where all the action is in the Apps store. There is a lot of competition to make sure that on the day you launch your app you get all the right coverage lined up and all the stuff happening.

People feel like they don’t want to do a bad launch under their real brand name, because that will harm their ability to do the proper launch later and get to the top 25.

But there is no law that says you can only bring out products under one brand name. I strongly, recommend that to people if you are very concerned about your precious brand. I think most startups are way too concerned about the power of their brand. They should be so lucky to get some kind of brand going.

Even still, bring it out under a terrible name. I specifically recommend people bringing products out under brand names that they hate so that they won’t ever be tempted to make that into their real official brand name and then become afraid to experiment with it.

You have got to be bringing products out under a brand that you feel comfortable experimenting in. Then once you find the right formula, there are two possibilities.

Either you will be able to port that product over to your new brand name and it will be great, or the product concept you brought up under that bad brand name will be so powerful, you just can’t get people away from it.

It is too sticky and you are stuck with it. But congratulations! You are successful! Is it really so bad that you personally don’t like the brand if customers do like it? I think it is not so bad.

Running pricing experiments in public

Nivi: A friend of ours has a popular subscription based product that they don’t charge for and now they want to start charging for parts of it for the premium model, and they want to find the optimal pricing strategy. How can they run those experiments in public and in secret? What would you suggest to them?

Eric: I would actually not be afraid to run them in public. It is hard for people who are afraid of what the worst possible thing that could happen is, to do this. But I think it is good to just try it and get over it.

What happens is, it is true that customers don’t generally like the idea that one customer got charged one price for something that somebody else got charged a different price for the same item. So there is some risk when you do different pricing offers in a split test.

But in my experience, there are two mitigating factors that make it not so bad. The first is it is actually incredibly difficult for most customers to figure out that is happening, especially if you only do it in a limited time window.

For example, I am going to tell you a story that may not seem related, but bear with me. When we were at there.com, the virtual world company, we would do a lot of QA. That was a heavy QA company.

For hours every day we had QA testers sitting in a lab together running the virtual world software and testing to make sure that it worked. I remember one day getting called in to see about…

There was one tester. They were around a physical corner from each other. So you couldn’t see each other, but they were not more than 20 feet away. They were both engaged in this activity.

The guy called me down and he said, “I am in this dune buggy riding around with somebody in the virtual world and we are seeing this glitch. We are not seeing the same thing. Something is not right.” They were calling back and forth, trying to pin down what it was.

I remember sitting there really confused about what the problem was, because it looked like the two of them were sitting there in the dune buggy and everything was fine.

I walk around the corner to the other guy. I talk to him about what the problem is. I look at his screen. On his screen, he and the other guy are engaged in a paintball match. They are not in a dune buggy at all.

He was almost a mile physically distant in the virtual world from where the other guy was, yet their conversation was perfectly consistent to them and it never occurred to anybody to ask, “Wait. What planet are you on in this time that we are comparing notes?”

They had no clue that this was happening. I think, we totally tend to underestimate just how powerful the pull of what you see is to most people. They basically can’t imagine the world, any other way than the way that it is.

Entrepreneurs don’t have that problem, so a lot of times they don’t grasp what is true for customers. It is actually very unusual for the customers to go onto a forum and post, “Here are all the offers that I am being offered and exactly what I see. Does anybody else see the same thing?” Our natural assumption is that everybody else sees the same thing.

So you are not totally likely to get caught. That is a mitigating factor. It is actually not as bad as you think when you do get caught, because don’t forget; you have the power of the apology, especially as a startup.

If you screw up… You are going to screw up all the time. If you are a customer of a startup, your general experience is, “These guys are constantly screwing up.” What customers care more about than whether you screw up or not, is how you treat them when you do screw up.

They care that you listen to them and take them seriously more than if you always get it right. If they want to work with a company that always gets it right, they will go work with some premium giant company that really has a very carefully constructed customer experience.

If you get caught doing this thing, you can always say, “We are so sorry. We were experimenting with this pricing. We didn’t mean for this to treat anyone unfairly. And if anyone was treated unfairly, we have gone back in the records and we are going to give them all double the money back for the thing that happened,” or whatever you have to do to make it right.

That is OK! It is really not that bad. What happens then is people say, “Wow. These guys are serious about making sure that we get treated fairly.” Meanwhile, you get to keep experimenting.

Opening board meetings to the entire company

Nivi: Yeah. You have talked a couple times on your blog about how you opened the board meetings up to the entire company and the positive benefits of that, and people’s perceptions of negative benefits.

Eric: Yeah. Well this is not something that a lot of companies adopt. This is considered pretty crazy.

I don’t know if it’s that most people are actually afraid of giving the whole company information they need to do their job, because it might lead them to judge the top management harshly, but people judge you harshly whether you give them the information or not, from my point of view.

Just give them the information! Your pathetic attempts to hide what’s happening don’t fool anybody.

Having been on both sides of that divide, I can tell you I never felt like I was being successfully fooled. And if you do manage to fool me for a limited time, I’m awfully pissed. My point of view is: you want people to have the maximum information possible.

You need to do it in a trust-building way. You’ve got to make sure the people you’re giving it to understand what they can and can’t do with that information, and they understand that they need to keep company secrets confidential.

If you don’t trust your employees to keep company secrets confidential, you’ve got bigger problems and you should go address those problems first.

There is some board business that has to be done in secret for legal reasons, so it’s not true that absolutely every meeting that any time ever happened at the board level is open to the public.

Employees have critical things to say in board meetings

Eric: But the interesting part about board meetings is the strategy conversation where you present progress, show data, and you make discussions about what should happen next. And that’s the part of the meeting I strongly recommend people open up to their employees.

What we did is we actually had a board of advisors and then a board of directors that was a subset of those advisors. We would convene the full set, advisors and board, at nine o’clock in the morning and we would have a maybe two-hour strategy conversation followed by maybe a half-hour or one-hour private board meeting.

For the strategy conversation the rule was: every employee can attend. We did this up until we were a 50- or 60-person company. We actually, physically crammed everybody into one room, and we had the employees sit around in as much seating as we could fit and the board members would sit at the big table.

It wasn’t a free-for-all, most of the employees were encouraged to listen, not to speak. But every once in a while the rule was that if someone had something they really needed to say, they could be recognized by the CEO and say their piece.

It was amazing. We would, occasionally have a board meeting where we would have a moment, where there would be data we were presenting to the board, and it would indicate that on a certain day, a certain metric went up and that was due to us launching that feature that day, or whatever our interpretation of what that data meant.

And not an insignificant number of times we would have an employee raise their hand and say, “Excuse me, but do you also realize that something else happened on that day?” Yada, yada, yada.

And occasionally, I’d be the one presenting! On the one hand, I’m really embarrassed. So I’m like, “No, I didn’t realize that.” This is a critical thing about running my own business I didn’t know.

But once I got over my personal embarrassment, what you would find is the board loved it! They’re like, “Thank God that guy was sitting in the room and could enlighten us about that. That changes our interpretation of what this means.”

And quite a few times I think we saved ourselves months of work by coming to a realization of something way earlier than we would have, because the right guy happened to be sitting in the room.

And yeah, occasionally you had an employee who’d make an off-color comment or say something that really shouldn’t have been said in front of the board, but people learn from those experiences. Most of the time most people had really substantive conversations.

Nivi: Did you ever get in a situation where some of the employees were like, “I don’t even care about these board meetings. I don’t even want to go?”

Eric: Yes, yes! We eventually had people who on occasion would beg me not to have to go to the meeting. And we eventually made them voluntary. For a while I was really rigorous, I said, “No, everyone has to be there. If I have to be at the meeting, you have to be at the meeting. Why do you think I’m any more privileged or unprivileged than you?”

Yeah, because board meetings are actually pretty boring. But when people are outside the room looking in — and you know most conference rooms have some form of glass — people can see what’s going on. They’ll come up with an excuse to walk by, kind of peek in. They will make up whatever crazy conspiracy theories are consistent with the data if they’re not there.

An, in my point of view, that’s such a source of waste: people gossip and there’s rumors and people don’t know. Let them be in the room, let them see how boring and mundane most board meetings are. So that for the occasional one where something actually interesting gets decided, let them be there to hear it themselves.

Everybody in the company has the ability to understand what everybody else in the company has to understand

Eric: There are some costs, definitely some down sides to doing it. One which took us by surprise was that, people can occasionally get confused about who’s in charge, we did occasionally have people — some board member would say, “You guys should really build feature X.”

Board members occasionally would just spout off about what’s randomly on their mind, and occasionally you’d have an employee get confused that that means the company is now going to go do feature X because board member so-and-so said so.

And that was actually good practice for us, to be a constant reminder that no matter who you are, no matter what it says on your business card, nobody gets to decide randomly that the company’s going to do feature X. Right? I don’t care if you’re the CEO or the lowliest person, we’re going to have a reasoned and considered process for deciding what to do.

Nivi: And it’s a learning opportunity.

Eric: It’s always a learning opportunity. The other thing that was hard for me personally was it’s hard to have your people who work for you see you be criticized in public. That was not fun.

Nivi: Hard for whom?

Eric: Well, it’s hard for me. My emotional reaction was like, “Wait a minute! I’m doing the best I can and now you’ve got to watch me get smacked around because I screwed something up.” But once I got over my personal emotional response to it, it was wonderful.

Because it humanized me to the people who worked for me — they got to see, “Oh, I see the pressure that he’s under” — but more importantly, when I needed something from somebody for the purposes of presenting to the board I could go to them and say, “Do you remember what happened the last time I didn’t have the right answers to these questions or I had shoddy this? You’re really going to send me in there with this? Come on, you’ve got to help me out!”

So it made us collaborators in creating solutions for the board rather than I’m constantly asking them for stuff and they don’t know why.

And I think, get over your own infallibility. We all make mistakes and it’s better for people to see what the real stuff is.

Nivi: I think you wrote about this on your site, basically the assumption is that everybody in the company has the ability to understand what everybody else in the company has to understand.

Eric: That’s right.

Nivi: The assumption is I have the ability to understand what the CEO has to understand.

Eric: That’s right. And that makes people uncomfortable, because sometimes we would say, “You have the obligation to understand what the CEO’s going through right now, because it’s going to impact the way you do your job.”

Some people would say, “I just want to sit in my narrow corner, do my little thing, and I don’t want to worry about what the company strategy is.” And we would show those people the door. We were really serious about that.

You really needed to have people who were… they didn’t have to be good at it! We weren’t asking them to be good at the CEO’s job, but we are asking them to understand why is the CEO making the decisions that he’s making. Because they’re going to have to make CEO-level decisions sometimes.

Sometimes the actions that have the biggest impact on the company’s performance are taken by people at the line employee level. They may not realize it’s going to have that big impact, but they are going to make those decisions. By the time the CEO finds out about it, sometimes it’s way too late to do anything about it.

We sure hope that the guy at the line level understands what the company strategy is and how his decisions impact, at least the best that he can.

Nivi: Yeah, I think maybe their decisions impact the company more than the CEO in the sense that if the CEO doesn’t come in to work, who cares? The company proceeds, but if the team doesn’t come in to work nothing happens.

Eric: OK, let me tell you: when the community manager takes a day off, you can have serious, serious meltdowns in the community if it happens to be the wrong day. That can have major impacts on the company.

Should we share bad news with employees?

Eric: I’ll say one more thing because this is a real effect that people are afraid of, which is that if you give people information about how a company’s doing, it can impact morale negatively. Sometime the company’s not doing well.

It makes some people have this idea that part of your job as a manager is to shield people from bad news or shield them from chaos. Because it’s not fair to them to have them have to do their job and also be confused about how the company’s doing. I just think that’s a really paternalistic attitude that we just need to let go as an industry.

If you want people to believe you when you tell them the good news, you have to sometimes tell them bad news. Otherwise, you have no credibility. And when there’s bad news to be shared, yes, it negatively impacts morale. But for a good reason, because things aren’t going well and we now need to rally the company around the fact that we need to change what we’re doing.

And there’s nothing like actually seeing the board say, “You guys have a major crisis on your hands that you have not yet understood,” to get everyone in the company saying, “We’re alarmed. We need to do something about it.”

That can cause some chaos, and that can be disruptive, but if you build trust and rapport with your employees then what you could do is you can sit everybody down for an analysis meeting after the board, which we would always do, and say, “OK, let’s talk about what we heard and what does it mean for the company,” and let people share their perspectives.

Let people say stuff like, “This says to me we need to cancel all our projects and completely retool.”

You need to get that idea out in the open because when somebody thinks that, you don’t want them to just unilaterally go execute on that plan! You want the opportunity to tell them and everybody else who didn’t have the courage to say the same thing: “No, we’re not retooling, but we are going to make some adjustments and here’s how we think about it, here’s what we’re going to do about it and here’s what’s going to happen.” That was pretty powerful.

Nivi: Thank you!

Eric: You’re welcome.

Nivi: I think that was great.

The minimum viable product (MVP) is often an ad on Google. Or a PowerPoint slide. Or a dialog box. Or a landing page. You can often build it in a day or a week.

I recorded the interview and synchronized it with some simple slides below. That’s my favorite way to consume the audio. You can also find a transcript and stand-alone audio below. Eric also highlighted some excerpts from the conversation on Lessons Learned and put together his own presentation on MVPs. Let me know what you think — I’m especially interested if you like the synchronized audio and slides.

In Part 2 of the interview, we discuss opening board meetings to the entire company.

Slides: What is the minimum viable product? (pdf)

Audio: What is the minimum viable product? (mp3)

What is the minimum viable product?

Nivi: First of all, this is Nivi from Venture Hacks, and I’m talking to Eric Ries from… where are you from?

Eric Ries: From the Lessons Learned blog.

Nivi: From the Lessons Learned blog, formerly from IMVU, formerly an advisor to Kleiner Perkins. We’re just going to have a discussion on a few topics of Eric’s and my choosing.

Nivi: First topic: What is the minimum viable product? Talk to me about minimum viable products.

Eric: OK, well let’s start with the question. Why do we build products in the first place?

In the end, we hope to be able to launch product to lots of customers and have them give us money so that we build a great business.

One approach to solving that problem would be, let’s build a product with the maximum number of features that will maximize our chance of success in the end. But the problem with that is you won’t get any feedback until you’ve already built all those different products.

All those different features, so you ship this product with a ton of features, and generally, by the time it’s done, it’s too late to make sure that you are on the right track.

The alternative would be, let’s do the release early, release often thing, and let’s get feedback as we go. The issue there is, if you just follow the release early, release often mantra, you find yourself running around in circles, because you ship code, you get some feedback from people, you do a focus group.

Some customers say, “Give me feature X,” “Give me feature Y,” now you’re kind of like, maybe sometimes you do what they want, maybe sometimes you’re going to do what you want, and then they get mad at you, and you’re chasing your own tail a little bit because you’re not operating against a clear, long-term vision of what you’re trying to accomplish.

The idea of minimum viable product is useful because you can basically say, look, our vision is to build a product that solves this core problem for customers, these kind of general feature areas, and we think that for the people who are early adopters for that kind of solution, they will be the most forgiving.

And they will fill in their minds the features that aren’t quite there if we give them the core, tent-pole features that point the direction of where we’re trying to go.

The minimum viable product is that product which has just those features and no more that allows you to ship a product that early adopters see and, at least some of whom resonate with, pay you money for, and start to gave you feedback on.

Nivi: So there is some set of customers out there, we believe, that just with these features alone, this product is useful to them.

Eric: Exactly right. And sometimes it’s useful to them because early adopters have the same kind of visioning power that entrepreneurs do, but because they can see what the end product is going to be.

Getting developers on the IMVU platform with a MVP

Eric: There are cases where you ship them a product that actually doesn’t work very well — at IMVU, the first version of our product worked pretty terribly, but, for example, it was all about user-generated content. We wanted to get developers on the platform.

The problem with developer platforms is this chicken and egg problem. Developers don’t want to develop unless there are customers who are there to buy their products, and customers don’t want to come on the platform unless developers are there selling them something useful.

What we did is we took early adopter developers and we told them a story about how IMVU was going to take over the world and be this really powerful product for mainstream customers and we made them believe it.

And we gave them an economic incentive that said, the earlier you get on board with the platform, the bigger your take is going to be for derivative products that get created down the road.

We shipped a product that basically had almost no customers — certainly no mainstream customers, and the developer tools weren’t that great — but, because we had told that story effectively and we really understood those early adopter developers, we got a ton of them on the platform developing.

Because they felt like they were in the middle of a gold rush, despite the fact that there was really no evidence to support that belief other than their own power of imagining what this thing was going to be down the road.

Luckily, we delivered on that vision, and so they actually were — a lot of them — pretty happy.

A non-MVP: The Kerry vs. Bush avatar

Nivi: I want some examples of these crazy minimum viable products. By which I mean, for example, and you can talk about any of these, the AdWords approach, the approach you talked about in terms of dialogue boxes and just popping those up, and just trying to sell a PowerPoint slide to an enterprise customer.

Eric: Let me start with an example of a time we didn’t know the minimum viable product, to illustrate my point.

It was 2004 — you have to remember 2004, Bush versus Kerry election. At IMVU, we had this idea — it happens to entrepreneurs all the time, you wake up in the shower and you’re like, “I’ve got an idea for a killer product.”

The idea for us — this was probably September/October 2004, debates are happening, politics is in the air — our idea was, we’re going to sell presidential debate avatar set that we would either dress up like Kerry or Bush, and you’ll be able to debate with your friends in this 3-D presidential debate product.

Now, put aside for a second whether that actually seems like a good idea to anybody else. We convinced ourselves it was a great idea, and we spent two full weeks racing to get this thing built, because it was time sensitive, the election is coming, the debates are happening, every day counts.

We thought it was a great idea for that whole two weeks. We built it out and we shipped it and we spent countless hours debating exactly what features had to be in it or not in it, and we were going to sell it for $1.99.

Our theory was, pricing won’t get in the way of anybody buying this thing. We want to make it cheap and easy and it will make lots of buzz and Wall Street Journal and New York Times are going to cover this thing; it’s going to be awesome.

I remember for us in those days, two weeks of development was a lot, because we were a pretty fast team. We did that, we shipped it — cut to the chase, nobody bought it. We sold exactly zero copies of the Kerry vs. Bush avatar.

We tried a bunch of different permutations and different variations of it and added features, and we changed the price, and eventually we gave it away for free, and even at free we couldn’t sell any copies.

Nivi: Well how many bought it for free?

Eric: None.

We literally couldn’t give this thing away. This thing was a dead weight money loser. There was actually nobody in AmErica interested in having a presidential debate avatar.

How to turn the Kerry vs. Bush avatar into a MVP

Nivi: How would you have approached this exact same product by taking the minimum viable product approach?

Eric: Well it’s interesting. We thought we were taking the minimum viable product approach because we had only spent two weeks on it. Right? Where we had made a very early prototype and put it out there.

But, if you think about it, going back to the definition of the minimum viable product, which is the minimum features that are required to learn what customers want, we had spent way too much time on it.

What we should have done, and what we did for a lot of features thereafter, is started with a landing page that promised people that product. Then we should have taken out the AdWords we were planning to take out, drive traffic to that landing page, and offer people to buy the experience that we are talking about.

What we would have found out if we were doing that experiment is 0% of people would have clicked through, which means it doesn’t matter what is on the second page.

The first page is so bad, not because it is badly designed, but because the features are wrong that you don’t need to go through the effort of building out the product. So we wished we had done that, and we did make that mistake really.

Nivi: How would you have marketed that minimum viable product to your existing customers?

Eric: When you already have customers you need to have some way to experiment with making them an offer. In a lot of cases the minimum viable product really is just that offer.

You can crisply articulate to customers what they’re going to get and how much they’re going to pay for it. You can learn a lot by just popping up a dialogue box that says, “Hey, would you like this new feature?” or showing them a banner ad for that feature.

For example, on IMVU we would have a system setup so that we could arbitrarily from the server select a small percentage of customers and make them an offer by inserting a dialogue box into their conversation, and basically it’s a simple “Yes” “No,” would you like this thing, “Yes” or “No?”

When they say “Yes,” we take them to a landing page where we try to sell it to them. Eventually if they really did want it, we would have to make up some excuse why we couldn’t give it to them like, “We’re experiencing technical difficulties right now.” “We’re not quite ready to give it to you. Give us your email address and we’ll email you when it’s ready.”

Again, you’ve got to remember that 99% of the time nobody wants it. Most offers that appear to an entrepreneur as a good idea are actually horrible, horrible ideas. By making the offer and having it be rejected by customers, we learn not to waste time building stuff that nobody wants.

Nivi: Right. Maybe the right definition of a minimum viable product, like you were saying, is, essentially a test to see whether people will actually want the product that you’re imagining in your head.

Rejecting false negatives: “But my customers don’t know what they want!” #

Eric: That’s right. The reason why this is an art and not a science is, I’ll have entrepreneurs come up to me and say, “But hold on, my customers don’t know what they want. If I ask them ‘Do you want this thing?’ They might say no when the answer is really ‘Yes.’”

Unfortunately, that’s an excuse that is used way too often, but there are situations where it’s true. The judgment call is; what really is the minimum set? In some cases like in entertainment products it might actually require you to build an early prototype, or a mockup, or even version one of the product with the minimum possible set of features that you think could go.

The nice thing about minimum feature set is you can always try intermediate points to ask yourself, “Am I at the minimum feature set yet? Am I at the minimum feature set yet?”

As long as you’re not afraid of the false negative, that is, if you don’t get discouraged because you’ve built your first paper prototype of it and shown it to people and nobody wanted it. That can’t mean that you give up because, “Oh, forget it, we’ll never make it.” You’ve got to say, “OK, well then let’s iterate some more.”

If you keep iterating at it, you keep making it a little bit more sophisticated, at a certain point after you’ve been through 10 iterations, that you still got no uptake whatsoever, and the feedback you’re getting from customers is still a yawn, you might say to yourself, “You know what? We’re not moving in the right direction. In fact, we’re past the point of minimum viable product. This just isn’t a viable product.”

Nivi: Right. Back to your point about entertainment, in Hollywood they start off in scripts. If the script looks good, then they will make a pilot. If the pilot looks good then they might order a few episodes from the first season. After the first season, then they make from there. They don’t build all three seasons, and then try to ship them.

Eric: Yeah, which means that some great shows never get made, because the early tests look negative and the people involved don’t have the courage or stamina to see it all the way through.

On the other hand, sometimes people have the courage and stamina to see through a really bad idea. That’s why the concept of learning is so important. This cannot be done on a spreadsheet. You have to keep training yourself on multiple iterations and multiple attempts to start to develop good instincts for it in your particular domain, your particular market space, what’s likely to work and not work.

Then you can do that as an entrepreneur, but even more powerful is if you can get your whole organization, everybody, training themselves constantly to do that kind of learning, so that good ideas will be passed around the whole organization.

Building products like packets get routed on the Internet

Nivi: Also minimum viable product ties into the concept of “build it before you sell it.” Are there any other out of sequence things that you guys did at IMVU or elsewhere that you found helpful? For example; you guys probably wrote tests before you coded to some degree as well, didn’t you?

Eric: Yeah, that’s right. I mean test driven development is the same principle.

Nivi: I’m wondering if there are some other things that you guys have done.

Eric: If you think about the full range of the product development life cycle; from specification to design, to implementation to testing, to maintenance to sales to deployment.

Nivi: Deployment and all that stuff.

Eric: I now think about that rather than as a linear sequence, I actually think about it as a big network. The question is; just for any given feature, in what path should you route it through that network? Different features should be routed differently, just like on the Internet we route packets differently as necessary.

Nivi: I’m wondering if you have some specific examples of paths you’ve taken.

Eric: Yeah, we just talked about writing tests first. We’ve talked about selling first. We’ve talked about deploying first.

Deploy first, code later

Nivi: When do you deploy a product? When have you deployed something before you built it?

Eric: All of these forms have offers of products. For example in a lot of Internet products, how does the customer know that the feature exists? Yes, there might be 100 pages of fancy stuff, but how do they even know that stuff exists?

Generally, it’s because there’s a link in the header. There’s one link on one key page that notifies them that they can go do this other thing or at a key moment they receive an email, even though the feature might be wide and complicated.

Generally, access to the feature is controlled through a simple choke point. We would often add those choke points in a split test just to see first of all, did anybody click on them? So we could look at the click-through rate of people that now believe there is this feature.

We would also see an interesting phenomenon, sometimes the presence of a feature, even if nobody clicks on it, still impacts their behavior in other ways.

For example — I can’t remember the data, we saw an experiment where, this isn’t exactly right but something to this effect, we added a link to the header notifying people that there was a VIP Club for special people only to get access to on IMVU.

Now, it’s not the same as what we have now. IMVU does have a VIP club today that is not anything related to this. But, in those days, the idea was, we were trying to test whether people wanted to have some kind of premium experience.

What happened was few people actually clicked on that link to go find out about the VIP club, but in the experimental cohort for people who were shown that header, their average spending was higher.

By constantly reminding them that there was such a think as a premium experience, we primed them to want to do more spending on IMVU. It was a very unexpected result.

And it’s a good example of why you always do full cohort-based split tests. Always test for the macro effect, don’t look at the micro effect. That’s a case where deploying a feature before we’ve built it actually can give you an insight that maybe you don’t need to build a feature after all.

Nivi: Interesting. Yeah. That reminds me of an example that one of our friends told us about where they couldn’t get people to sign up for a free product, so they put a paid version of the product right beside it, and I don’t think anybody signed up for the paid version…

Eric: It made the free version look a lot more…

Nivi: Yeah. It made the free product look legitimate. And they started to actually convert on that basis.

Design first, code later

Eric: There are also cases where you want to design a product last, rather than design it first.

We would often ship things in a schematic form with horrible design to see if we’d gotten the information flow and information architecture right, and really good interaction designers, if they’re being honest, will tell you that that’s always the sequence.

You always re-factor your design out of specific use cases and out of specific uses, rather than starting with the broad vision of what it’s like.

Nivi: Basically the trails where the people walk, you pave those?

Eric: Exactly right. And you could do that — I remember one time we used to use paper prototypes where you have designers sit down with a focus group of people and show them a fake screen shot that they came up with in Photoshop.

And then the customer says, “I want to click that button,” and you go grab the piece of paper that represents what happens when you click that button, you show them another prototype, and we used to do them that way.

We actually built a paper prototyping website into our website where, for certain customers, the designer could come in and actually replace sections of the website with full-on mockups from Photoshop that were sort of clickable and sort of functional.

Then we would bring a customer in and have them register an account; we would specify that that account should see the paper prototype, and then we would go through with them, do the focus group.

But the customer doesn’t know they’re seeing something that’s a mockup. They think they’re using the product.

And, yeah, the product is a little weird, because sometimes they click things and nothing happens, and sometimes things won’t look a little right, but customers think that about your product anyways, so it’s not particularly abnormal for them to see a product that looks completely incoherent to them. That’s the state-of-the-art for most customers and those products.

That was another case where we would — and we would, occasionally, come up with some prototype that that functioned well enough that we would ship it to real customers while we’re working on the full on, blown, beautiful version of it.

Sometimes customers would see a schematic version from which we would gather more information about whether we’re on the right track, and put things in the right place.

Where did you get the money to experiment?

Nivi: What allowed you guys to do all of this experimentation? Specifically from a financial point of view? Not about the mentality behind it and so on. Time is short and money is running out, or were you basically break-even this whole time, or how did you work that?

Eric: No, we believed in a concept we called “bridge to profitability, ” which is, whenever we raise money, we always raise money on the basis of a plan that we honestly believe, would get us to profitability on just that round.

If you notice, that doesn’t mean that we actually got to profitability on that round. In fact, we kept raising more rounds because we kept believing that we had the opportunity to pursue a bigger opportunity by raising money, but we never had to raise money. We always felt like we could get there on our own.

But that meant that we were constantly a dollar short, because plans to get to profitability on a round generally involve you going negative and then catching up before the money runs out, so we were not eager to learn, eager to experiment.

We did not love metrics. We were very scrappy and very afraid of running out of money, so we only ever adopted practices that we thought would have a high ROI.

The issue is that these practices do have a high ROI, and we got to them because we kept following the traditional model and failing and feeling like, God, we are burning money like there is no tomorrow. We keep building features that nobody wants. We literally don’t have the time and we can’t afford to keep doing the same old thing over and over again.

I don’t know if we were just lucky that we had that mindset, but, for whatever reason, even though at some level we always believed the next feature we shipped was going to save the day and have a step change in our profitability and take us to the moon, because it’s just the best feature since sliced bread.

Because we were so desperate to actually make money, we — I’m trying to think of how to put this — we became obsessively committed to actually being right rather than just believing in the next thing to save us.

If you look at the traditional hockey stick shaped curve, it’s flat for an awful long time. We had been in a company previously where we were always promised, “Yeah, it’s flat now, but we’re on the verge of the hockey stick.”

The problem with the hockey stick plan, when that’s your plan, is you just work for a year and then the 13th month you’re going to go to the moon. For the 12 months, you don’t know if you’re making progress or not, because you’re following the plan.

Everything is flat like it’s supposed to be, because when the final feature comes into place, you’re going to go to the moon.

We just didn’t believe that was going to happen. So we forced ourselves to actually test each feature to see if it was going to be successful.

Then we just stopped being able to drink our own Kool-Aid, because we were just wrong so often, it became harder and harder and harder for us to convince ourselves that it was a good financial investment to just randomly try the next new thing versus actually trying to say, “Hey, is there a way that we can increase the probability of having a successful feature.”

Nivi: Yeah, and I think, there are probably two other things here, right? You charge from day one, so basically every experiment you ran was an experiment to make more money, right?

Eric: That’s right.

A preview of part two

Nivi: Why would you not run experiments if you’re charging from day one. And then, the second thing is — this gets us to the second topic, which is: You guys weren’t doing any of this really in public because you had not launched the product, right?

Eric: Amen.

Nivi: Nobody knew who you were. But people, at the same time were using the product, so how did you do that?

“Investing in startups seems mysterious and difficult. How much are you supposed to invest? What legal agreements do you need? Where do you find startups to invest in? How do you pick winners?”

People liked Naval’s presentation — Dave McClure called it an “awesome fucking talk,” and other tweets seemed to agree. So we thought you might like it too — it’s good advice for investors and entrepreneurs alike.

Paul Graham from Y Combinator wrote up his talk in a wonderful essay called How to Be an Angel Investor. So we’re calling our presentation “How to be an angel investor, Part 2”.

Slides: How to be an angel investor (pdf)

Audio: How to be an angel investor (mp3)

Transcript

Here’s a transcript of the conversation. I’ve highlighted some of my favorite bits.

Nivi: Hi, this is Nivi from Venture Hacks.

Naval: And this is Naval from Venture Hacks.

Nivi: Naval recently gave a talk at Y Combinator’s AngelConf on how to be an angel investor and we thought we would record it — people liked the talk — and we thought we would record a slightly longer version and put it online.

Naval’s going to do most of the talking and I’m going to jump in with a question or a comment.

Naval Ravikant

Naval: So, I’ve been in this business for a little while now, been involved with starting about seven companies, been an investor as an angel or a VC in well over 30 companies and been an advisor to over a dozen companies, and of course co-author Venture Hacks with Nivi.

After a while, you do enough of this and you start pattern matching — you notice a few things — and so I’ve put those together here and I’m using those to help out people who may want to get into the angel investment or even early stage venture business.

What do you need to be an Angel?

Naval: Basically, to be in the angel business you absolutely have to have three things — to be in the business and to do well. First of course, is access to capital: either you own or you manage to raise the funds or somebody just gives you money.

You need good dealflow and by good dealflow, I mean, proprietary or a high-quality dealflow that not everybody has seen or you have some unique ability to get in to. There has to be some reason why people will take your money instead of somebody else’s. Otherwise at the end of the day, you are commoditized and you get bid out of the game.

Finally, you need good judgment. Access to capital you either have or you don’t. If you don’t have it, then you probably shouldn’t be in the angel business, you’re just wasting people’s time. If you have good judgment, everybody thinks they have, and it takes about 10 years to figure that out so I can’t really help you there.

Dealflow

Naval: So, what I’m mostly going to focus on is dealflow, which is how do you find good dealflow, how do you process it efficiently, how do you get into good deals, how do you evaluate them? When it comes to dealflow, there are a lot of bad kinds of dealflow and there’s some good dealflow. Early on, you will be seeing very few deals and you’ll just be trying to get in the dealflow.

After you’ve made a few investments, you’ve got a few entrepreneurs who know what you’re doing, you have built a bit of a brand, you’ve connected to the other angels you will have too much dealflow, and you will quickly have the opposite problem. If you raise your hand and you say to the world, “I can write a check”, the world will beat a path to your door.

I’m going to help you with an odd problem, which is helping you filter the bad dealflow as well. So, you’re going to get a lot from your social friends, these are the people whose kid goes to school with your kid or someone you just hang out with.

Most of this dealflow, although you should be polite and you should see it, you’re going to have learn how to say no without taking a meeting, otherwise you’re going to find your time chewed up very quickly because these people are not in the business, they don’t have the same filters that you do, and they will pass along anything that looks like a company that’s raising money.

You, yourself, are going to be tempted to do the same thing and forward deals to other angels. Don’t ever forward a deal you wouldn’t do yourself.

Nivi: Or you’re not seriously considering.

Naval: Yes. It’s going to hurt your credibility, and it’s also going to also waste the entrepreneur’s time. The first thing any serious investor will ask you if you, yourself, are an investor and you forward a deal and say ‘”Are you investing?” You don’t want to hem and haw at that point.

You either want to say “Yes, I’m seriously considering it and I’m in the middle of my evaluation process” and you should mean it or you should actually be doing it. Those are kind of the only two legitimate answers.

If an entrepreneur asks you to forward a deal that you have passed on, your best approach is to politely explain to them that, it does not make sense for you to do so as it would send a very negative signal.

Nivi: Yeah, and if you’re an entrepreneur in that situation, I would actually ask for some names of potential investors and tell the angel that just turned you down that you’re not going to use their name to reach out to these people, but you just want to get a list of names to find some other prospective angels to reach.

Naval: Yeah, that’s a very good point. A lot of angels know a lot of other angels, you get a list of names then you have to sift through your own resources and frankly, you browse through your LinkedIn and your Facebook networks. If you’ve been around in Silicon Valleyfor a little while, you can usually find someone who knows someone.

Finally, one issue that crops up a lot when people first get in to investing is, they think they’re going to add their value in some unique process-oriented way, and so they come up with non-standard models. One could be incubation; another one could be acting CEO or so on.

They kind of say, “OK, I’ll put in some money and then I’m going to be super-active, and I want this and that and I’m going to take more stock into that.” I would suggest not doing that. Certainly you can offer it to people you know well, and certainly people can ask you for additional help or these kinds of non-standard deals, but they have two problems.

One is if you end up taking too much equity early on. It makes the company difficult to fund by VCs down the line. Secondly, it will lead to an adverse selection case, where if you’re putting out your services as a rent-a-chairman or a rent-a-CEO, you’re going to see the companies that are so desperate raising financing where they won’t entertain that notion.

Most really great entrepreneurs, yes they could use some advice and they’ll listen to it, but they don’t need you to run their company for them. So be careful about that adverse selection trap and be prepared that with some of the very, very best companies, your only value-add is essentially will be your money plus a few connections and a little bit of advice here and there.

What’s good? Deals that come from other angels who are doing them are usually very good. You build these up by quid pro quo. If they send you deals, you send them deals back and vice versa.

Branding your advisory services is very good. So if you have particular domain expertise, you are the founder of a company that was successful in a certain space, then you should be able to brand yourself in such a way that you will see the companies in your space.

Another way to brand yourself would be through some kind of a horizontal play. If you’re a marketing genius for consumer web companies, then people may come to you for help on that.

Nivi: Yeah. So, here are a few examples of that. The way that Naval put it at AngelConf was Paul Buchheit is the Gmail guy; Naval, himself, is the Venture Hacks guy and he’s also an expert on viral marketing and Facebook, and Paul Graham is Paul Graham. Those are the kind of examples you want to try to follow.

Pitches

Naval: So, let’s assume you’ve worked out this incredible dealflow pipeline and you’re getting a lot of companies that are coming to you. You want to get through them relatively efficiently. What you will find very quickly is that most business plans, executive summaries are a waste of time.

You can’t sign NDA’s, because doing so exposes you to far too much down the road. So what you really need is very high concept pitch where somebody explains to you, “we are this for that” like an analogy…

Nivi: Friendster for dogs.

Naval: Exactly. Dogster. Maybe a short 10 slide deck. If somebody comes to pitch you, don’t schedule hour-long meetings, you don’t need an hour to figure it out.

Usually within 20 minutes, is enough for the company to get their points across and you should not be pitching yourself at this point. First, you should just be listening; you can always pitch yourself a little bit later.

Nivi: I would say a high-concept pitch, like Friendster for dogs is great. One paragraph, or a couple paragraphs elevator pitch in email form is great, and if the company has it, a 10 slide deck. Of course, more important than any of that is a working demo or screen cast or mock-ups or some realized version of the product.

Naval: Generally, it’s a poor sign if a start-up comes to you, especially if they’re on the web, or consumer web, or iPhone, or one of these spaces where it’s pretty easy to hack something out and they haven’t put together a prototype or they haven’t, as Nivi would point out, made early and meaningful contact with the customer.

Nivi: Which can be done through things like surveys, as well, right? If its offline and it involves hardware or whatever, they could have gone out and talked to the customer via surveys or interviews, or they could have spent the last 10 years of their life working with that exact customer and know them inside out.

Evaluating Startups: Team

Naval: So, now when it comes to a valuable startup, you’ve got the short pitch, you’ve taken the half-hour meeting. There are times when you can eliminate the startup on the basis of team; this is not, obviously, fool-proof. Investing is not a science, it’s an art.

But here are some general rules of thumb I’ve developed over the years, which helped me, filter dealflow. One of those is that usually the optimal configuration for a company, in terms of the founder basis, two to three founders and three is the max.

Five is a very unstable configuration, eventually it will fall apart. You cannot have five cooks in the kitchen.

One is too difficult. The person gets too lonely and it gets too difficult to continue on, because they don’t have a thought partner. So two to three turns out to be ideal.

If you look at the very, very successful companies in the Valley, Apple had Steve Jobs and Steve Wozniak; Microsoft up in Seattle had Bill Gates and Paul Allen and later on Paul Allen dropped out, they had Steve Ballmer and of course, Google had Larry Page and Sergey Brin; even Oracle had Larry Ellison and quickly thereafter had Ray Lane.

So generally, you need teams of two founders at least. You want someone in the company who can build and someone who can sell and if there’s going to be an imbalance; it should, probably, be in favor of the builder.

Of course, it depends on the exact company, but you would rather be investing in a startup that has two builders and one salesperson, rather than have two sales people and one builder.

Generally, you want to say no to companies where the founders are all lawyers or all business guys and they don’t have the product development expertise to back them up or where they are outsourcing the product development completely.

Companies that outsource product development completely, generally have a hard time iterating and keeping up with the changes in the market and the competitors.

One learning that I’ve found the hard way is that is that, especially in consumer web deals, traction matters more than anything. It’s very hard to predict mass consumer behavior. It’s a very efficient market out there.

So, it’s good to find real customer adoption before you put too much money to work, whereas in enterprise deals, where you are selling to a small number of large customers, the team can make a meaningful difference.

Nivi: One thing on the topic of a guy that can sell; it doesn’t have to be your conception of a prototypical salesman or a car salesman or whatever. If you are in a one-on-one meeting with Bill Gates, even though he is a complete nerd, you would be extremely compelled by whatever he has to say to you.

So in that sense he is a salesman, right? A lot of nerds in their nerdiness can actually be extremely compelling in the proposition that they are trying to put in front of you.

Naval: Yes, by salesman, I don’t mean the bag-carrying enterprise sales quota matching guy. It’s exactly as Nivi said, people who are compelling and who can convince early customers to sign up, through whatever means possible. They can convince early employees to sign up when the company has very little cash. They can convince early investors to sign up.

In fact, usually if someone can sell you in an investment meeting really well, and you walk out dizzy, saying I don’t really get the product or this space, but this guy almost sold me. That’s a good sign.

Nivi: Yes, that in and of itself is an asset in terms of being able to sign on customers, employees, other investors, and so on.

Naval: One other thing to think about is when you are evaluating companies at an early stage, the appropriate title for the people in the company is usually founder, or engineer, or things of those sort.

If you see a company that’s very top-heavy, has its CEO, its CFO, VP of this, SVP of that, then it’s usually a bad sign. It means that the people involved are more interested in building up impressive titles for themselves or building a large organization than they are in having a successful product and a good financial outcome.

Nivi: Yes, we want builders.

Naval: In evaluating entrepreneurs, Warren Buffett often says that when he looks at the company he buys, and very often these are multi-billion dollar companies, he’s looking for intelligence, energy, and integrity. I find that the same criteria works extremely well in evaluating start-up founders also.

Intelligence for obvious reasons.

Energy because drive and passion are the number one predictor of an entrepreneur’s success, and it’s not actually intelligence in that case.

And integrity is paramount, because if you have someone with high intelligence and high energy, but they don’t have high integrity, you’ve basically got a hard-working crook. You don’t want to be associated with that. In fact, you prefer lazy, dumb crooks to hard-working smart crooks.

Nivi: Yes. They will have many opportunities to screw you, if they don’t have any integrity.

Evaluating Startups: Customers et al

Naval: So, intelligence and integrity I would not compromise on any of the three.

Similarly, we talked a little bit about teams. Let’s talk about the customers. As we talked about earlier, you don’t want to invest in companies that have made no contact with the customer yet whatsoever.

For an enterprise software company, it might be surveys; it might be a few big customers’ meetings references. For a web company, it usually means the product has been released or some facsimile of it has been released and tried out.

You definitely want to look at a lot of companies, before you make early investments. The usual failure mode for individual angel investors is that they invest too early and too quickly. Part of this is you don’t have a partner who can veto you.

Venture funds are deliberately organized, in such a way that there are a lot of experienced people saying no to the new guy’s urge to go invest in the first hot thing he finds.

In angel fund, I would suggest that you be disciplined either by numbers, so say to yourself, I’m not going to make an investment until I see at least twenty companies. One hundred might not be a bad number. In a busy year, I’ll easily see four, five, six hundred companies.

An even better approach might be to find some experienced angels that you team up with and give them an effective veto. No matter how much you like the deal, if you can’t talk them into it, don’t do it.

Nivi: Yes, and along those lines, you could just say, I’m not going to invest in any deals that these five notable angels are not also investing in for my first couple of companies.

Saying No

Naval: So, saying no is something that you are going to be doing far more often than saying yes. So, let’s get that out of the way. You have to learn how to say no, you have to learn how to turn down people.

It’s uncomfortable, but usually people appreciate a quick and up-front and reasoned no, with some good advice and hopefully a little bit of help attached to it, than they do a dragged out long process where at the end then you hem and haw and have to say no.

So, the best time to say no is before you take a meeting. If some of these deals fail your filters, or if you just can’t see yourself getting excited about the company or the business, say no over email before you meet.

If you hear the pitch, and it is not compelling, take five minutes out after the pitch, talk about it, think about it, and then say no to the entrepreneur before they even leave.

Since angel deals tend to be pretty small, you are investing tens or hundreds of thousands of dollars, you don’t want to drag it out over months. You don’t want to take weeks of the entrepreneur’s time. You don’t want to cause them to generate additional documents or write code just for you.

So try and be efficient about it. Don’t spend more than a couple of hours of their time, or if you had to noodle on it yourself for more than a few weeks, then it probably isn’t right for you.

Warren Buffett makes two really good points. Actually, Ron Conway lists one of them at the angel conference, which was don’t invest in people that you really aren’t excited about working with, because at the end of the day, the business model will change. The details will change. The company’s progress will change.

The only constant will be the person you invested in. If you don’t like to take their phone calls, if you don’t like to sit down there and talk to them all the time, then eventually you are going to see the deal as a burden.

You will eventually run out of time. So, it’s very important that you only pick the people that you want to work with. I just can’t emphasize that enough. It sounds like an easy thing to say, and hard to follow, but in reality, you will find that it will make all the difference in the long term.

The other thing that Buffett points out is that this is a game where the swings that you don’t make are not counted against you. You just have to find a few hits, and if you let a lot of hits go by while you are waiting for the perfect pitch, that’s absolutely fine. So, when in doubt, just say no.

Since the angel community is very small, the entrepreneur community is very small, your reputation is everything. So, it’s very important that you try and add value to every company whether you invest in them or not.

If they took the time to present to you, and they paid you that honor, then you should take the time to maybe make an introduction or two for them, or give them some useful, honest advice. Generally, it’s not a good idea to try and withhold your capability to help them too much, just because you did not make an investment.

Navi: Yes, and that is where you area of expertise, whether it is raising money or marketing or Facebook or whatever, can really shine right at the end of a meeting.

Saying Yes: Terms

Naval: So, let’s talk about the terms for a moment. So, let’s say you’ve processed a huge amount of deals, you’ve found a company that you want. You’ve said no to the ones that you don’t want. You want to make an investment. Some things to keep in mind is that one of the biggest risks of angel investing is the downstream financing risk.

You may look at a company and say, I could see this being a small but highly likely exit. That’s not initially a great company for you to invest in. The danger there is that a lot of companies use up cash very quickly, they run out of cash. You don’t want to put money after your own investments, because you are obviously now a biased investor.

And VCs always evaluate things in terms of their huge market potential, as VCs have very large funds. It ends up being a little bit of what the economists call the Keynesian beauty contest, which is you pick a winner based on who you think the judges are going to pick to win in the next round.

So, you are not initially picking the most beautiful start-up, you are picking the start-up the judges are most likely to pick. So, it is a dead-end trap to finance too many companies that don’t have the possibility of turning into something huge somewhere down the line.

One way to mitigate that is to co-invest with other venture capitalists. This is where you’re an angel, you’re putting a small amount of money and the VCs are putting in most of the money.

The issue there is that VCs are ownership sensitive. They want to own a lot of the company, but they tend not to be as price sensitive as angels because they have lot of money to throw around. So, one way to mitigate that is to give the VC some different rights that you don’t care about, such as the ability to invest more money the next round, and in exchange keep the valuation lower for you as the angel of this round.

Another opportunity for some superstar angels, or if you are taking a very active role in the company, is you might be able to get a little additional advisory stock, relative to the amount you invested.

Nivi: Just to be clear on Naval’s concern, his concern is that the seed round — that VCs get involved — ends up being for a lot of cash coming in at the high valuation. That makes it economically uninteresting for you as an angel investor, because you will end up owning a very small slice of the company because the valuation will be high.

So his suggestion, if I understand it right, is you basically give the investors — you do the round at a low valuation, you have a reasonable amount of cash coming in, the VCs co-invest, and what you give them is basically an option to invest in the next round. Is that right?

Naval: That’s correct. And in fact, another point of the economics is a lot of first-time angels, especially when investing their own money, tend to invest in what are called “uncapped debt.” So they’re put in a convertible note at a 20% discount to whatever the next round is going to be.

That’s an extremely uneconomic proposition. The odds of a startup going out of business are far greater than 20%, or the 16% that would be required to cover your investment. In fact, there are corporate bonds trading in the public markets today that are yielding 15-16%. So they are far more liquid with companies that are far better capitalized.

Generally, you’re looking for a 3X, 4X, 5X increase in valuation between rounds, which means you are price sensitive. So it’s OK to do a convertible note because that’s often simpler for everybody involved, from a legal perspective. But you generally want to establish a conversion price cap, which pays you back for the risk that you’re taking at this stage.

Nivi: One thing, just going back to the VC thing, from the perspective of an entrepreneur, if I have VCs investing in the seed round, the issue that it creates for me is that if those VCs don’t want to invest in the series A, it makes it very difficult to do the series A, because the inside investors who knows the company best are basically saying the company is not a venture investment.

Naval: One other point about investing with groups. A good reason to invest with other angels, no matter how much you like the company, is that stuff always goes wrong with a startup. And when stuff goes wrong, very often the entrepreneur can make the choice to walk away, especially if the company has a very small amount of money invested.

Or the entrepreneur can choose to do something that would be adverse to you, because you’re a minority shareholder without a lot of the protections that VCs often have. If you’re in a group of angels, you are much more likely to have a good outcome in that situation.

An entrepreneur is much less likely to walk away, if there are five good angels as investors than if there’s only one. An entrepreneur is less likely to change the terms that are adverse to you, if you’re with four other angel investors rather than if you’re alone.

One way in which, if you’re an angel investor, you can squander a lot of time is if you’re board seats. Essentially if you’re taking board seats, you’re going to be limiting yourself to only being able to handle five, six, seven investments, which may not be enough for you to achieve the portfolio affect that you need for the economics to work.

If 1-out-of-20 to 1-out-of-30 angel investments work, and you can only make five because you’ve taken board seats and your time is full, then your economic model is in trouble, and you would have to augment it by sprinkling around a lot of companies where you almost spend no time with them.

Board seats are really only worthwhile taking if you’re getting paid for it, if you have some unique and proprietary insight to the company and it’s market — maybe you started a company in a similar space that did really well — or in that unique case where the company is just someone who is very, very close to you.

Otherwise, realize that boards spend a lot of time on corporate governance issues, which often isn’t the best use of your time.

Nivi: Yeah, I mean, you can simply advise the company. You don’t have to be on the board to advise the company. The other downside is if you’re not willing to leave the board whenever the company asks you to, you can screw up their series A investment, because VCs don’t want to come into a company, which has a random guy who was one of the angels on the board of directors.

Naval: Yeah. It depends on who the angel is, of course.

Nivi: It depends, yeah.

In terms of how much of the company you want to buy, you’re not going to end up with 40% or 50% of the company after an angel or seed round.

You should be thinking more in terms of 5%, 10%, 20%. Capital that is coming into the company is $25K, $50K, $100K, up to $250K, the valuations that you’re looking at are going to be in the $0.5 million to $2 million dollar post-money valuations.

The way you justify the company’s valuation isn’t on the basis that the company is actually worth that much money, in the sense that somebody would buy the company for that price today. The way you justify it and the way entrepreneurs justify it, is through dilution.

You need to leave enough of the company in the hands of the entrepreneurs who are the guys who are creating value for you and for themselves alike, to be incented to create value for the two of you.

If after the first round they own less than half the company, and there are three more rounds of financing coming along over the next five years, they’re not incented to create value for you or for themselves.

So that’s where you end up with the angels or the seed investors owning somewhere between 5% and up to 20% of the company after a seed round.

The other way entrepreneurs justify the valuation is just through a market clearing price. Things are worth what people pay for them, and as long as someone is willing to pay $2 million post money for a company, you have to pay the same price, either if you want to invest alongside them or you want to outbid them.

Economics

Naval: So let’s talk about the economics a little bit. Think of yourself as a patron of the arts. If you were in 16th or 17th century Italy, you would be investing in paintings and sculptures. You’re doing the same sort of thing now. You’re not really in this so much to make money, as you are to help the next batch of entrepreneurs get to where you got to.

It’s unlikely that most of us as entrepreneurs would have been successful and gone up in value, without angels as investors, and now you’re giving that opportunity to the next young batch. And I say that because it’s very difficult to make money in this business.

The hit rate is extremely low; the economics are tough because on the really interesting companies it gets bid up by the venture capitalists.

And very often even if you find a hit company and you manage to get a piece of it early on by making and investment, very often you can get wiped out if there’s a bump in the road down the line and you don’t have the capital to pony up.

So think of yourself as a patron of innovation. You’ll sleep a lot better at night, if you assume your investments are lost on the day that you make them. And I’m not the only one to say that. At AngelConf, I think most of the professional angels, including the ones who had been in it for a long time like Ron Conway, made that point.

And your personal portfolio, outside of your angel investments, should be balanced with ultra-safe investments. Nassim Taleb, the author of “Fooled by Randomness” and “The Black Swan,” advocates having a 10% ultra-risky portfolio, in other words, looking for a positive “black swan.” And the other 90% being in T-bills and cash and equivalent safe investments.

But it’s an extremely rewarding business in the sense that you get to work with very, very bright people who are young, full of energy and fire. They show up and they tell you everything they’ve learned about a topic, that they’re deeply passionate about, in a short period of time. So the learning curve is high.

It’s extremely fun, but if you’re not careful you can lose a lot of money and annoy people.

Nivi: I don’t think that I’ve been involved in a company that somewhere along the way was not catalyzed by an angel investor or a seed stage investor. I don’t know if that’s the case for Naval or not.

Naval: I have, but usually the ones that had good angel investors worked out much better. In the angel case, as an entrepreneur, you can get the advice and help and connections that you look for out of a VC but you can get it with much less dilution and much less capital and, most importantly, much less control over your company.

If you’re really looking for good advisors, it’s usually a much better idea to go and get a couple of angels than it is to raise a big VC round. If you need a lot of capital, because you have something that you’ve figured out is working and you want to scale it, that’s when venture investment makes sense.

Nivi: Yeah. One way to think of yourself as an angel is, as basically an advisor that pays for the privilege of advising a company, and in the process gets a little chip in the business, in the case that the company happens to be a Google or a sub-Google like exit, Skype, MySpace or whatever.

Naval: And to echo a point that Paul Graham made a long time ago, the best, highest quality people that you want involved in your startup are usually already pretty successful. You’re not going to be able to recruit them as employees, you probably won’t even be able to bring them on as advisors for small amounts of equity.

The only way that those people will participate in your businesses is, if they have a meaningful stake.

And you don’t want to give huge stakes to a person just for a few words here or there where you don’t know what the value will be down the line, no matter how famous or how successful they are. So the way the market clears is they get to invest in your company.

Venture Hacks

Naval: If you have any questions about this or if you would like to talk more about it, feel free to contact us at founders@venturehacks.com. We’d also like to know if you’re an angel investor who’s available to invest capital and what your profile is.

So feel free to email us with that and we will try to show you interesting companies. What we would ask is that you tell us how much capital you have to invest, how much you’re looking to put to work, and what your particular areas of expertise are.

Podcast-lovers can also receive Steve’s course as a podcast. Just use your podcasting client to subscribe to our feed:

feeds.venturehacks.com/venturehacks

If you use iTunes, this link should work automagically. If it doesn’t, cut-and-paste the URL above into iTunes like this:

Step 1

Step 2

The podcast also includes any audio or video we post on Venture Hacks, such as Eric Ries’ presentation on lean startups.

Class 2: Three types of markets

Our first post about Steve’s course included two classes (and two MP3s). Some podcasting clients can’t handle two MP3s in a single post. So we’ve moved the second class and its MP3 here.

Audio: Customer Development 2: Three types of markets (mp3)