“Decisions tend to be judged solely on the results they produce. But I believe the right test should focus heavily on the quality of the decision-making itself…
“Individual decisions can be badly thought through, and yet be successful, or exceedingly well thought through, but be unsuccessful, because the recognized possibility of failure in fact occurs. But over time, more thoughtful decision-making will lead to better overall results, and more thoughtful decision-making can be encouraged by evaluating decisions on how well they were made rather than on outcome…
“It’s not that results don’t matter. They do. But judging solely on results is a serious deterrent to taking the risks that may be necessary to making the right decision. Simply put, the way decisions are evaluated affects the way decisions are made.”
Rubin looks at the world through the lens of probabilities. A good decision might have bad results. But if you have the chance to do it all over again, under the same conditions, you should make the same decision.
Evaluating advisors
“Any time you make a bet with the best of it, where the odds are in your favor, you have earned something on that bet, whether you actually win or lose the bet. By the same token, when you make a bet with the worst of it, where the odds are not in your favor, you have lost something, whether you actually win or lose the bet.”
Evaluate advisors on the quality of their decision-making, not on the quality of their past outcomes. The same goes for advice from investors, or anyone at all.
Entrepreneurs and investors can make poor decisions and still succeed. They can get lucky. But the odds that the same thinking will work at your company aren’t favorable.
Other startups can make great decisions and still fail. They can get unlucky. But their thinking wasn’t bad — they just need to roll the dice again.
Evaluating decision-making
“Simply put, the way decisions are evaluated affects the way decisions are made.”
This is the coolest part of Rubin’s speech.
First, we evaluate decision-making processes and pick one. Second, we execute the process and get a decision. Third, the decision leads to action, which leads to an outcome. (All of these steps can be iterative.)
Does anyone actually evaluate decision-making? Yes. Often it’s implicit. For example, a job interview is an evaluation. A candidate has a decision-making process whether he knows it or not. And an interview evaluates a candidate’s decision-making process whether a startup knows it or not.
The decision-making process you pick is more important than the decisions it produces. And the way you evaluate processes is more important than the process you pick. Evaluation is king.
Evaluate decision-making processes based on their quality, not on a handful of their outcomes. For example, consider whether job interviews are an effective way to evaluate a candidate’s decision-making. In other words, (evaluate (processes for evaluating (processes for making decisions))). In other words, duh.
Learn more: Be the House
# Sean Murphy · Jan 15, 2009
The concept of “good decision, bad outcome” is a key one to master. To often we infer the quality of the decision from the results alone. This can lead entrepreneurs in time periods like 2004-2007 to think they are better at making decisions than they are and to doubt themselves too much in a downturn.
# Robert J. Fischer · Jan 16, 2009
I think that the good decision bad outcome paradox constantly plagues stock market investors. Good poker players understand what Skansky meant, but it seems that investors are much less likely to understand this. Could this be because investors have the illusion of control and that poker players understand that the cards won’t always fall their way?
# James Rohrbach · Jan 18, 2009
Nice post. I’d be interested to push this into more practical (vs. theoretical) territory. What are common “processes for making decisions” and “(process for (evaluating (processes for making decisions))))” among entrepreneurs, and which are/seem to be more effective?