After a non-technical team has already established product-market fit as defined by Sean Ellis* – for a service with a browser/app interface that wants to start scaling.

Let’s presume that this late joining technical founder gets competitive salary a…

First, I am curious to find out how you deal with the fact that online surveys (especially with a small number of responses) are not representative of your overall user base. Usually, people with extreme views will be more likely to fill it out. Does the 40% rule incorporate a level of bias in the sample?

Second, when you suggest that the business model will come out of this early user feedback, I presume you don’t mean that a startup in a pre-market fit stage should have no hypotheses about their business model? How much can you plan before you find your market fit?

We are providing an enterprise SaaS and our target demographic – pharmaceutical and biotech outsourcing – numbers in the hundreds of thousands potential users. Given the relatively small number (and specificity) of our audience, we don’t have the need to rev on hundreds of pages. The solutions are quite unique and specific to our user’s needs, therefore we don’t encounter the challenges associated with building a landing page which appeals to a broad audience.

In my experience, given our targeted audience, we have found that after 5 revisions, we have come pretty close to our messaging. Beyond that the benefits become marginal.

To test the various landing pages, we approach it in two distinct phases. The first phase consists of designing the landing pages and testing them with a small group (10~20) of our “most active” users. Once we cull the designs down to 2 landing pages, we then conduct some A/B testing to see which ones results in greater conversions. In our case the conversion metrics are lower bounce rates and maximizing sign ups. I have found it difficult to quantify difference between 3 versions and send the users evenly (50%/50%) to the A/B versions. The key idea here is to make the A/B versions different enough, so that you can clearly identify why a user selected one page over the other.

When we started out, our initial user base was painfully small. We were lucky to leverage our relationships and networks to find folks who would be willing to try out our site. In addition, since we identified a clear pain point, it wasn’t subsequently hard to find early adopters to try and test out our solution. If I could have done things differently, I would have conducted the “must have” survey as soon as we released our working prototype – one month post launch. If for nothing else, to figure out if we were on the right track. I would conclude by saying that given our small target user base and clearly identified pain point, your task is somewhat easier – than say, a consumer application, as you can come relatively quicker to identifying messaging that resonates with your audience.

Read “4 Steps to Epiphany” by Steven Blank. If you haven’t already done so, this is a must read, before you design or build anything.

Hope this helps. Best of luck!

Jae, in your case, it seems your site isn’t the kind that would have some crazy amount of hits. So how do you fork off people to try the different landing pages? i.e. Do 80% get the old one, 5% the new one, 5% another new one, and so one?

Sean/Jae, I ask, because I’m working on an enterprise SaaS site, where ultimately I’m not expecting a lot of traffic, and I’m struggling with how to do the landing page test. Not why to do it and not how to get 20 landing pages built, but how to actually fork off the people in the right manner.

Similarly, for a fully new site in enterprise software, where I have no existing customers, I’m in a situation where people would be given trials, we’d work the trial, and close the deal. However, the total customer base could be incredibly small, so at what point should I be doing the survey about must-have?