Customer Development Posts

I watched most of the Startup Lessons Learned Conference from home. Thanks to the magic of justin.tv, I also brushed my teeth, had breakfast, cleaned the bathrooms, and did a couple phone calls at the same time. Here are my two favorite talks from the conference.

Steve Blank: Customer Development 2.0 “Why Accountants Don’t Run Startups”

Steve‘s talk is a must-watch. I’ve watched it twice now.


Video: Customer Development 2.0

Kent Beck: Beyond agile programming

Kent Beck is a natural speaker, brilliant, and humble to boot. Fans of agile software development and extreme programming will especially enjoy this talk. At least watch until he talks about scratching goats — that’s my favorite part.



Video: Beyond agile programming

There were many other excellent talks at the conference — see them all here. Now I’m looking forward to the Startup School conference, coming later this year.

In Where’s the demo?, I asked, “Unless you’re famous or a big company, why do a closed beta?” There are lots of good reasons why in the comments.

But my favorite comment is from Rob May, founder of Backupify and AngelList power broker:

“In the early days of Backupify (when it was still called Lifestreambackup), not only was I worried that someone would steal the idea and execute it better, but I was also embarrassed by the quality of the product when I demo’ed it.  But ultimately, some early users loved it and that kept us going.  By releasing an early buggy minimum viable product, we got a pretty big lead on everyone else.  Now we have a bunch of competitors, but with more than 60K users and 30 terabytes of data backed up, we are pretty far ahead in terms of traction. Being open early worked well for us.”

By the way, Backupify recently raised money from General Catalyst and First Round Capital.

Thanks to KISSmetrics for supporting our interview with Sean Ellis. If you want an intro to KISSmetrics, send me an email. I’ll put you in touch if there’s a fit. Thanks. – Nivi

Hiten Shah from KISSmetrics recently sat down with me to discuss how to optimize funnels with their upcoming analytics tool, KISSmetrics.

You may know Hiten from his Crazy Egg days and survey.io, which Hiten and I discussed in How to measure product/market fit with survey.io.

SlideShare: How to optimize your web apps with KISSmetrics
Audio: Interview with chapters (for iPod, iPhone, iTunes)
Audio: Interview without chapters (MP3, play anywhere)
Transcript: See below

Prerequisites

You’ll get more out of this interview if you also read:

  1. Our interview with Sean Ellis parts 1 and 2.
  2. Our previous interview with Hiten: How to measure product/market fit with survey.io.

Outline

Here’s an outline and transcript of the interview. The interview and transcript are about 23 minutes long and we’ve highlighted some of the juicier bits for you.

  1. After fit, prepare for growth
  2. KISSmetrics helps you optimize your funnel
  3. KISSmetrics helps with all optimization steps
  4. First user experience isn’t necessarily the paid user experience
  5. KISSmetrics is most valuable after fit
  6. How Cloudfire uses KISSmetrics during fit
  7. Startups aren’t a science — but we’re getting closer
  8. How other people use KISSmetrics
  9. KISSmetrics is on it’s third iteration
  10. Why KISSmetrics’ hasn’t launched
  11. KISSmetrics tracks actions on a per-user basis
  12. KISSmetrics lets you calculate customer LTV
  13. Get more startup advice

[Read more →]

This guest post is by Ash Maurya, a lean entrepreneur who runs a bootstrapped startup called CloudFire. If you like it, check out Ash’s blog and his tweets @ashmaurya. – Nivi

What you charge for your product is simultaneously one of the most complicated and most important things to get right. Not only does your pricing model keep you in business, it also signals your branding and positioning. And it’s harder to iterate on pricing than other elements of your business. Once you set a price, coming down is usually easier than going up.

Should I charge for my MVP?

Most people choose to defer the “pricing question” because they don’t think they (or the product) are ready. Something I hear a lot is that a minimum viable product is by definition (embarrassingly) minimal. How can you possibly charge for it?

A minimal product is not synonymous with a half-baked or buggy product. If you’ve followed a customer development process, your MVP should address the top 3 problems customers have identified as important and it should do it well. You can ensure that by dedicating 80% of your efforts to improving existing features versus cranking out new ones.

Steve Blank bakes price exploration right into the initial customer interviews. Price, like everything else, is built on a set of hypotheses that needs to be tested early. Steve suggests you ask potential customers if they’d use the service for free. This is to gauge if the product’s value proposition is compelling at all. You then ask if they’d use the service for $X/yr. How do you come up X? You can simply roll the dice and adjust along the way, or use Neil Davidson’s excellent guide to software pricing to start with a more educated guess. Once your MVP is built, Steve asks you to sell it to your early customers. There is no clearer customer validation than a sale.

Sean Ellis, on the other hand, argues that achieving initial user gratification (product/market fit) is the first thing that matters and suggests keeping price out of the equation so as not to create unnecessary friction:

“I think that it is easier to evolve toward product/market fit without a business model in place (users are free to try everything without worrying about price). As soon as you have enough users saying they would be very disappointed without your product, then it is critical to quickly implement a business model. And it will be much easier to map the business model to user perceived value.”

Both Steve and Sean advocate removing price from the equation — but at different points. Steve removes price during the customer discovery process but suggests you charge for your MVP. Sean removes price from the MVP and suggests you charge after product/market fit. I can see the merits of both approaches and wondered which was right for my product: CloudFire: Photo and Video Sharing for Busy Parents.

Why not use freemium?

On the surface, freemium seems like the best of both worlds: Get users to try your service without worrying about price, then up-sell them into the right premium plan later. However, many people make the mistake of giving away too much under the free plan, which leads to low or no conversions. It’s human nature — we all want to be liked.

More important, we don’t yet have enough information to know how to price or segment the feature set. I made this mistake with my first product, BoxCloud: an early visionary customer called me up and said, “I really like your product and want to pay for it but your pricing doesn’t require it.” After a few more iterations of segmenting the feature set, I decided to forgo the free plan and simply offered premium plans with a trial period. Sales went up and so did the quality of feedback, which I attribute to the difference between feedback from customers versus users.

(Hiten Shah shared a similar story with me around his experience with Crazy Egg. Even 37signals has greatly deemphasized their free plans to almost being fine print on their pricing pages.)

Lincoln Murphy just published a timely white paper on “The Reality of Freemium in SaaS” which covers many important aspects to weigh when considering Freemium, such as the concept of quid pro quo where even free users have to give something back. In services with high network effects, participation is enough. But most businesses don’t have high enough network effects and wrongly chase users versus customers. What I particularly liked in this paper is Lincoln’s recongition that “Freemium is a marketing tactic, not a business model.”

I strongly feel that, especially for SaaS products, starting with free and figuring out premium later (all too common) is backwards. If you know you are going to be charging for your product, start by validating if anyone will pay first. There is no better success metric and it leads to less waste in the long run. Focusing on the premium part of freemium first lets you really learn about your unique value proposition — the stuff that will get you paid. You can then come back and intelligently offer a free plan (if you still want to) with more intelligence and the right success metrics clearly defined. Even if you think you have a one-dimensional pricing plan like I did (e.g. number of projects), you’d be better served testing it with paying users because pricing experiments take a much bigger toll than other types of experiments.

Testing price in interviews

How did I put all this to test? The biggest mind shift in following a lean startup process is going from thinking you know something to testing everything you think you know.

So I followed Steve Blank’s advice and built some pricing questions into my initial face-to-face customer interviews. Because CloudFire is a re-segmented product in an existing market, potential customers referred to competitor pricing. This had to be balanced against the perceived value of our unique value proposition – saving time with faster and easier sharing of lots of photos and videos. Through these interviews I determined that, like their sharing needs, my potential customers valued simple hassle-free pricing and $49/year for unlimited photo and video sharing was a fair price they were willing to pay. That is what I charged them once my MVP was ready.

Testing price on the web

I wanted to run the same set of pricing tests with web visitors that I did during my interviews. Short of split testing a free and paid version of the MVP, which is technically illegal (update) and unfair to paying customers, I decided to split-test 3 different products with 3 different prices:

  1. $49/yr for unlimited photo and video sharing
  2. $24/yr for unlimited photo sharing
  3. FREE for 500 photos

All plans have a 14-day free trial with the exception of the free plan which is free forever. Here are the variations I tested:

Original: Single unlimited plan

This is the simple option I discovered during customer discovery interviews. It served as the control.

Variation 2: Multiple plans

I segmented the product into 2 offerings: unlimited photos+video and unlimited photos only. I wanted to test price sensitivity and gauge interest in video sharing. Not many people I interviewed were currently taking lots of videos but they all wanted to be taking more.

Variation 3: Freemium

This has the 2 plans from above along with a limited free plan. Yes, this is a freemium plan. I wanted to measure if a limited free plan would disproportionately drive the right type of traffic (busy parents in my case).

Variation 4: No Price During Introductory Period

I added a fourth variation to test Sean Ellis’ advice on removing price till product/market fit, but I tested this differently. I was not comfortable offering the full product for a price and for free at the same time. So rather than including this page with my A/B tests, I instead tested it with new parents I interviewed.

The Results

First Place: The original single plan — second place in conversions and best overall performer. Surprisingly, the original page was the best overall performer.

Second Place: Variation 3: Freemium – most conversions but second place overall. Not surprisingly, the freemium variation drove the most conversions but only outperformed the original by 12% and had the lowest retention. Referral stats combined with random polling/emailing revealed a majority of the users that signed up were just curious (and not parents).

Third Place: Variation 2: Multiple plans – least conversions and worst overall performer. People reacted least favorably to the two paid plans.

Non-starter: Variation 4: No price during introductory period. Parents I interviewed did not understand the introductory period without explanation and were reluctant to try the service without knowing how much the service was going to cost. Probing further, they weren’t willing to invest the time building up web galleries and inviting others only to find that the service might be priced out of their expectation.

What I learned

It does pay to align pricing with your overall positioning. Our unique value proposition is built around being “hassle-free and simple” and people seemed to expect that in the pricing model as well. A lot of our existing customers were already paying for their existing sharing service so the leap from free to paid was not a big one. While Sean suggests removing price before fit for consumer facing products, he suggests always charging for enterprise customers to gain their commitment. This is another case where pricing needs to be explicit. Using Cindy Alvarez’s model, our customers appear to be Time-Poor, Cash-Rich. Offering no-hassle free trials was sufficient to remove the commitment risk. Money back guarantees might be another way to further lower this risk.

The biggest lesson learned, though, is how accurate my initial customer interview findings were, compared to all the hypotheses that followed. Pricing is more art than science and your mileage will vary, but whenever possible get out of the building, talk to a customer, and consider testing price sooner rather than later.

What do you think? Why do you think these variations finished the way they did? What other variations would you like to see us try? How else do you think we could increase conversions? I’m looking forward to discussing your responses in the comments.

“Where the #@!*% is Part 2?”

That’s what I’ve been hearing since we published Part 1 of our rare interview with Sean Ellis.

Here’s part 2.

In Part 1, Sean discussed what you do before product/market fit: how to get there, how to measure it, and how to survey your users so you can improve fit.

In Part 2, he explains what you do after fit: optimizing your positioning, implementing a business model, and optimizing your funnel — all so you’re prepared to acquire users quickly and profitably.

If you don’t know Sean from his blog or tweets, he lead marketing from launch to IPO filing at LogMeIn and Uproar. His firm, 12in6, then worked with Xobni (Khosla), Dropbox (Sequoia), Eventbrite (Sequoia), Grockit (Benchmark)… the list goes on. 12in6 “helps startups unlock their full growth potential by focusing on the core value perceived by their most passionate users.”

This is the first time Sean has done an interview on the record. I’m really psyched he’s making his insights public — this interview is a must-listen.

SlideShare: How to bring a product to market, Part 2
Audio: Interview with chapters (for iPod, iPhone, iTunes)
Audio: Interview without chapters (MP3, works anywhere)
Transcript with highlights: Below

This inteview is free — thanks to KISSmetrics

We’re bringing this interview to you free, thanks to our sponsor KISSmetrics.

Sean is an advisor at KISSmetrics and we interviewed their CEO, Hiten Shah, in How to measure product/market fit with survey.io. KISSmetrics built survey.io with Sean — now they’re collaborating on KISSMetrics, a new tool for funnel optimization that we’ll discuss in an upcoming interview with Hiten Shah.

Prerequisites

You’ll get more out of this interview if you also read:

  1. Part 1 of our interview with Sean.
  2. The startup pyramid.
  3. We use several phrases interchangeably in the interview: Growth = scaling = acquiring customers with a known ROI. Preparing for growth = after product/market fit = optimizing promise + implementing economics + optimizing the funnel.

Outline

Here’s an outline and transcript of Part 2.

  1. What comes after fit?
  2. Prepare for growth
  3. a) Put the metrics in place
  4. b) Optimize the funnel
  5. c) Optimize the messaging
  6. Prepare for growth as quickly as possible
  7. Remove bottlenecks to preparing for growth
  8. Preparing for growth is not a low-burn period
  9. Preparing for growth doesn’t require growth
  10. Use a business model to grow quickly
  11. Grow and create new channels
  12. Nail the first user experience while preparing for growth
  13. Leave no room for the competition
  14. Marketplaces are an exception to this model
  15. Why Sean decided to focus on startup marketing
  16. Just scratching the surface

[Read more →]

David Anderson, VC and supply chain master:

“Billy is perhaps best known for his Oxi Clean commercials, where one softball size ball of detergent, chucked into your washing machine and forgotten, will do 25 loads of laundry. Simple messages, great customer testimonials and products that really worked were Billy’s forte. He rejected perhaps 99% of the hundreds of inventors/companies that approached him, choosing only those products that met his three key metrics. And he and his inventors made millions in the process…

“Address clear customer needs: Billy wanted products that anyone could use, not just a select market segment–who does not have dirty clothes (Oxi Clean)? Needs a knife to cook (Samurai Shark)? Want shiny clean floors (Orange Glo)? or, my favorite, who has ever wished they could permanently cement their mother (or father)-in-law to a chair (mighty putty)?

“Have believable customer testimonials: Billy did not use actors, but recruited users who were true believers. Who could not like the housewives who had orgasms over their newly waxed floors?”

(Italics mark my emphasis.)

That’s a great set of requirements for picking startups: simple messages, great customer testimonials, and products that really work — that anyone can use. (You can also take the opposite approach on the “products that anyone can use” requirement.)

Bonus: Also see David’s post on Hiring the Right Sales Guy:

“So what’s the best type of sales guy for a start up? To be honest, none of the above. The founders are the best sales guys, supported by the correct marketing and sales support technology, processes and people. They know their solutions and vision best and should be the one who lead the charge in the marketplace.

Instead of hiring a very expensive super sales guy, for example, use your board members to get introductions at target customers. That’s a prime role for board members and should be a key criteria in choosing them. Want C-Level introductions? Try cold calling at 8am in the morning when the C-levels get in early to check their emails. Need a group of specialists to sell the solution?  Assemble the team from current employees to make the pitch. Everyone should be a sales person in a start up.”

This is consistent with Steve Blank’s advice that founders should be on the customer development team and, if necessary, complemented by a ‘sales closer’.

David Skok, serial entrepreneur turned VC at Matrix Partners:

“In the many thousands of articles advising entrepreneurs on what they have to focus on to build successful startups, much has been written about three key factors: team, product and market, with particular focus on the importance of product/market fit. Failure to get product/market fit right is very likely the number 1 cause of startup failure. However in all these articles, I have not seen any discussion about what I believe is the second biggest cause of startup failure: the cost of acquiring customers turns out to be higher than expected, and exceeds the ability to monetize those customers… [emphasis added]

“A quick look around all the B2C startups shows that, although viral growth is often hoped for, in reality it is extremely rare. When it does happen, the associated businesses are usually extremely attractive, provided they have a way to monetize their customers. (For more on the topic of Viral Growth, refer to my blog post on that topic here.)

“Far more common is a need to acquire customers through a series of steps like SEO, SEM, PR, Social Marketing, direct sales, channel sales, etc. that will cost the company significant amounts of money. What shocks and surprises many first time entrepreneurs is just how high the numbers are for CAC [Cost to Acquire Customers] using these kinds of techniques.”

Read the rest of David’s excellent post: Startup Killer: the Cost of Customer Acquisition. But don’t worry about it too much until you’ve built a product people want. It’s hard to know what the CAC is before you know what the product is. And check out the rest of David’s new blog — his post on Viral Cycle Time is great.

The comments to our interview with Sean Ellis turned into an awesome Q&A — this post is a roundup.

(Feel free to keep asking questions here or there and I’ll try to get answers from Sean.)

Is my product a nice-to-have or a must-have?

Jae Chung wonders whether his product is a nice-to-have despite the positive press:

“I spent the past 24 hours poring over each of the points [in the interview]. We also formed about 8 months ago and the site is currently undergoing beta testing and has received positive feedback from many of our users and the press. However, my gut tells me we are in the “nice to have” category, and could never quite put our finger on what it was that users found appealing. We’ll definitely be implementing your survey to find out where the “love” is!”

The survey he’s talking about is survey.io.

Should I charge users before fit?

Sean Ellis:

“I think that it is easier to evolve toward product/market fit without a business model in place (users are free to try everything without worrying about price). As soon as you have enough users saying they would be very disappointed without your product, then it is critical to quickly implement a business model. And it will be much easier to map the business model to user perceived value.”

Michael Harry Scepaniak:

“…freeing yourself (pun intended) of paying customers early on would seem to allow you to make more radical moves (pivots), since you don’t have to worry about angering anyone that has given you money and expects you to deliver on their expectations in return.”

Instead of charging users for a part of the product they don’t even want, first find the part they love, and then figure out how to get users to pay for it. Entrepreneurs who advise you to charge from day one probably had fit early on in their startups.

How do I tell users that I’m going to charge someday?

Eric Santos:

“Do you communicate to the users that the product will have a price someday?”

Sean Ellis:

“I would communicate that “it is free during Beta” or if “beta” is too techie, then free during the introductory period. If you plan to have a free version, you can also let people know that.”

Should I pay users to send feedback?

Gregory:

“What about offering a gift or paying users to send feedback? Is this a useful technique, why or why not?”

Sean Ellis:

“I haven’t needed to offer a gift for feedback yet. However, on SMB targeted products I tend to create a formal beta program that includes feedback requirements. Those people that participate in the beta program lock in a discount on the product (generally I don’t announce price at this point, just that they will receive a 50% discount). In addition to providing great feedback, these people tend to convert at a very high rate (since they worked for a discount).”

What if my customers aren’t filling in surveys?

Vincent Chan:

“From my experience, many SMB users don’t like to fill in a survey for an unknown startup. Should I take that as a bad sign? In other words, is the survey response rate an equally important metric as the “must-have metric”?”

Sean Ellis:

“Yes, I’ve found survey response rate directly correlated to the percentage of users that consider the product a “must have.” For “must have” SMB products I often see the response rate over 10%.”

How do I find the love in a hardware company?

Samuel Bouchard:

“Sean, how can this “find the love approach” apply to hardware companies? What needs to be adapted to the method when you sell a product that is worth several $k’s?”

Sean:

“Samuel, I haven’t worked on a hardware product, so I’d just be guessing… Given the cost of getting a hardware product to market, I’d spend a lot more time up front on “where’s the need?” Steve Blank’s book Four Steps to the Epiphany gives great guidance on this.”

If you can build a product in a day, show customers the product. If the product is going to take weeks, show customers a PowerPoint instead.

Play us out

Ryan Nile knows what it means:

“This basically describes what we need to do after the MVP is up.”

Thanks to KISSmetrics for supporting our interview with Sean Ellis. If you want an intro to KISSmetrics, send me an email. I’ll put you in touch if there’s a fit. Thanks. – Nivi

Hiten Shah from KISSmetrics recently sat down with me to explain how to use their product, survey.io [update: this has moved to PMFsurvey], to measure product/market fit and find the “best grass” in your product. You may know Hiten from his Crazy Egg days.

survey.io is a free survey tool that helps you implement some of Sean Ellis’ techniques to get to fit. KISSmetrics actually built survey.io with Sean. Unfortunately, there hasn’t been great documentation for survey.io besides Sean’s launch post. Until now.

SlideShare: How to measure product/market fit
Audio: Interview with chapters (for iPod, iPhone, iTunes)
Audio: Interview without chapters (MP3, works anywhere)
Transcript with highlights: Below

Prerequisites

You’ll get more out of this interview if you also read:

  1. An example survey from survey.io.
  2. Our interview with Sean Ellis.

Outline

Here’s an outline and transcript of the interview. The interview and transcript are about 19 minutes long so we’ve highlighted some of the juicy bits to get you started.

  1. survey.io: Before product/market fit
  2. survey.io measures fit
  3. How did you discover the product?
  4. How would you feel if you could no longer use the product?
  5. What would you likely use as an alternative if product were no longer available?
  6. What is the primary benefit that you have received from the product?
  7. survey.io is open-ended
  8. The flock will always find the best grass
  9. You don’t need survey.io to find the best grass
  10. Have you recommended the product to anyone?
  11. What type of person do you think would benefit most from the product?
  12. How can we improve the product to better meet your needs?
  13. survey.io is more powerful with filtering
  14. Would it be okay if we followed up by email to request a clarification to one or more of your responses?
  15. Upcoming survey.io features
  16. Get qualitative feedback before fit
  17. Must-have % by industry
  18. Ask the must-have question

[Read more →]

Sean Ellis recently sat down with us and explained how to bring products to market. You should listen to this interview for ideas on how to get to product/market fit, how to measure fit, and how to survey your users so you can improve fit.

If you don’t know Sean from his blog or tweets, he lead marketing from launch to IPO filing at LogMeIn and Uproar. His firm, 12in6, then worked with Xobni (Khosla), Dropbox (Sequoia), Eventbrite (Sequoia), Grockit (Benchmark)…  the list goes on. 12in6 “helps startups unlock their full growth potential by focusing on the core value perceived by their most passionate users.”

This is the first time Sean has done an interview on the record. I’m really psyched he’s making his insights public — this interview is a must-listen. We’ve broken the interview into two parts: 1) before fit and 2) after fit. This post contains Part 1 (and here’s Part 2.)

SlideShare: How to bring a product to market
Audio: Interview with chapters (for iPod, iPhone, iTunes)
Audio: Interview without chapters (MP3, works anywhere)
Transcript with highlights: Below

This inteview is free — thanks to KISSmetrics

We’re bringing this interview to you free, thanks to the kind support of KISSmetrics. Sean is an advisor at KISSmetrics and we interview their CEO, Hiten Shah, in How to measure product/market fit.

KISSmetrics built survey.io with Sean — now they’re collaborating on KISSMetrics, a new tool for funnel optimization.

Prerequisites

You’ll get more out of this interview if you also read:

  1. An example of the survey.io survey Sean uses before fit. (Several phrases we use in the interview mean the same thing: Product/market fit = Fit = 40% of surveyed users consider the product a “must-have” = 40% of surveyed users would be “very disappointed” if they could no longer use the product.)
  2. The startup pyramid.
  3. Some of our favorite posts by Sean.

Outline

Here’s an outline and transcript of Part 1.

  1. Half the marketing battle is the product
  2. Must-have products make marketing much easier
  3. How PayPal built a must-have product
  4. Understand the must-have users
  5. How to get to product/market fit
  6. How not to get to fit
  7. Should I launch?
  8. You can’t set a deadline on fit
  9. How to communicate with the board during fit
  10. How to use positioning to improve fit
  11. It’s frustrating to try to grow without fit
  12. How many users do you need to determine fit?
  13. Who do you survey?
  14. What is promise?
  15. Don’t over-position your product during fit
  16. Pivot the business around the love
  17. Focus the product on the love
  18. Create a great experience around the love
  19. Create a business model around the love
  20. First find the love
  21. Are recommendations a good indicator of fit?
  22. The must-have metric is a good indicator of fit
  23. Preview of Part 2: What comes after fit?

[Read more →]