Nivi · March 31st, 2010
I just replied to a startup that applied to AngelList with a website behind a closed beta wall:
"Thanks Jeff. Where's the demo? If your team isn't pedigreed (founded a company and made money for its investors) and you don't have outstanding traction, the #1 thing you have going for you is a demo. I also think closed betas are generally a bad idea. Who cares if someone sees the site? Hope you don't mind the direct feedback. Send us a demo and we can take it from there."
Unless you’re famous or a big company, why do a closed beta?
Topics Pitching
Possible reasons:
— To avoid bad indexing in search engines (SEO view);
— You (still) use stock photos with no proper rights (legal view)
P.S. I know that there are robots.txt — but who knows.
EXACTLY. I’ve been saying this for a while now.
Your “coming soon” page should completely describe your service, otherwise I’m moving on by and not entering my email address (we’re working on bringing this to fruition now).
Similarly, your beta should do the exact same thing. I’d even argue that unless you’re twitter (or apparently Quora now…how they’re worth nearly $100M is beyond me), your alpha/beta should be as open as possible. It’s called customer development and MVP testing.
It’s now becoming a bit more mainstream, but our good friend Mr. Ries is all about the pivot.
A closed beta is a perfectly legitimate tactic for someone practicing customer development if it’s paired with a clear landing page and email submission. The point at the demo stage is not to grow customers, but just to measure interest and test the value of a minimal product. Having a closed beta lets you time the flow of new users with developments in your product.
If it’s not paired with a clear landing page, then the problem isn’t that the company is in closed-beta, it’s that the company is in stealth mode. I think the problem with the above closed beta is just that they didn’t include an invitation to the beta. FWIW, Mint was a closed beta from an unknown founder.
I agree with Tony. I’m definitely not a fan of the stealth concept. No one, more or less, wants to steal your idea.
But having a closed beta while you polish your product makes sense. Why risk 1) spending too much time on customer support with a < MVP product or 2) turning off potential customers? Not every product is fully baked, and I think it can make sense to fine tune with a limited user set. Also, I imagine it helps people get over shyness about releasing. Better to do a closed beta earlier rather than being tempted to wait for a better version.
Other reasons:
— The product is still a formless blob.
— The product is a business app, thus the usefulness threshold might be a higher than a consumer app.
— You’re narrowing your market and using beta feedback to make the right development/marketing decisions.
The only other reason I can think of is if you need to throttle access to test scaling, keeping two sides of a market in sync with each other, etc. Other than that I agree with Sean and Tony that if you’re not letting people in, you had better be telling them what they’re missing!
We launched to private beta just over two weeks ago with a beta code required to sign up and try things out. It didn’t take very long to scrap the “private” in our beta.
It ended up being a barrier to entry to try out a simple application and we saw lots of eyes hit the home page but quick exits when they saw a beta code was required (even with a simple request mechanism). Since scrapping the code, we’re seeing a lot more sign-ups and starting to get feedback from real users that we would have missed out on.
There are contexts and cases where maintaining a gate is warranted, but for most apps/sites getting started, acquiring users is hard enough. There is no sense in making it even more difficult for people to see what you’re about.
Agree with Tony, can think of a dozen reasons for having a closed beta site. It just depends on what the app is, who is the target market, etc. Not all sites are meant for just having traffic to build mass; sometimes it is not the quantity but the quality. Better to perfect to a small group than to expose a gap to a large one. If you are in stealth mode then by all means get feedback from those you invite — that is the reason for being in stealth, correct?
Getting customers is probably the hardest thing to do for a bootstrapped company. Why would you want to private your beta when you’re trying to get customers and get feedback? I think it’s self-defeating unless you have $$ to spend on PR/marketing to compensate for the missed users.
The point is that they probably DON’T want to scale their customer base, they need a selected few for QA, initial feedback, and a gazzilion other reasons to have a limited closed beta.
@David — This post wasn’t about bootstrapped companies, it was about investment-seeking companies. But bootstrapping-meets-lean startups is something I wish more people talked about. When you’re bootstrapping, cash is king. If they might pay, then you want their money, even if you haven’t reached the 40% love you category of value yet.
We decided to go straight from our initial signup page (used while we built our prototype) to launching a public alpha demo (at SXSW). This is probably one of the best early decisions that we have made. Instead of only being able to talk about our “promise” we were now quickly able to demonstrate our product vision to potential clients. Within a few days we had enough feedback to validate or invalidate some of the key ideas that we were playing around with. This helped us to prioritize our dev queue — now we know what our customers want, rather than guessing what they want.
It makes compete sense to run a closed beta if your business has high direct costs (Spotify) and you’re scaling your monetization in that market to match it (advertising sales).
Nivi — I understand why people feel that way. In the early days of Backupify (when it was still called Lifestreambackup), not only was I worried that someone would steal the idea and execute it better, but I was also embarrassed by the quality of the product when I demoed it. But ultimately, some early users loved it and that kept us going. By releasing an early buggy minimum viable product, we got a pretty big lead on everyone else. Now we have a bunch of competitors, but with more than 60K users and 30 terabytes of data backed up, we are pretty far ahead in terms of traction. Being open early worked well for us.
Some of these have been touched on already but…
1) Trade Secret protection. Probably not applicable for consumer/social media sites, but if you genuinely have protectable subject matter, trade secret law requires that you take affirmative steps to keep it secret.
2) You are using a beta for testing, not marketing. Once upon a time, software was tested in alpha, beta, and gamma releases before ever being foisted upon customers. If you know that things are going to break, but you still want to solicit feedback from fault-tolerant customers, a closed beta still works.
3) You want to stress-test infrastructure without throwing the doors open. Blizzard does this with many of their games. The early closed-beta is tested against one slice of their infrastructure with a finite and known number of customers. They can then extrapolate from these results and decide how much hardware to deploy at launch.
4) Perceived exclusivity. If you’re selling to a market where you aren’t relying on volume alone to drive profitability, the perceived exclusivity of a closed beta invite can be useful in building loyalty for premium-priced services.
Nathan — Excellent point on the trade secrets aspect — don’t hear much about that subject but I can tell you from painful experience that most start-ups do not understand what a trade secret is much less why they should be concerned about protecting their trade secrets. This could be one of the most important considerations in doing a closed beta because the sad truth is that it does happen and more than one would think.
All in all, enjoying the lively feedback.
Notwithstanding the reasons for keeping a beta private, if you’re soliciting investment, you should certainly have a beta key generated for anyone who needs to evaluate your progress.
It isn’t a clear decision. As an entrepreneur, you should be as focused as possible on removing barriers for your users until it starts hurting the business.
In this example, it’s possible the right move was not to scrap the private beta, but to include a login:pw for a demo account and a beta code.
Possible reasons (ignoring famous companies or founders) for a private beta:
— High COGS & no bizmodel (music, video, backup, etc.)
— Community quality is an important financing asset (quora)
— Legal issues (music/video primarily)
— Scaling concerns (if users are supposed to rely on your product for a frequent and necessary function of their job/life it often makes sense to only let in customers who understand the wheels might fall off occasionally. Esp. true in b2b plays, but a good consumer example = webmail).
We elected to go public with ours as soon as we felt it was stable enough to release. We are still working things out, but the user feedback has been invaluable. Ultimately we think the best thing for any tech startup is to just get it out there. In this industry if you wait, someone else will do it first.
It also makes sense if resources are limited and providing your beta testers a full/good experience of your product is a must.
I’m surprised Nathan only mentioned perceived exclusivity. A key part of customer acquisition is press coverage. The press is much more keen to write about your application if you provide only their readers exclusive access. Hence, keeping your beta closed and enticing MORE bloggers and journalists to write about you can lead to MORE customers in the long run.