Decision-Making Under Uncertainty: Fund Your Next Test, Not a Big Bet11 min read

Your review board has a EUR 1.2M, 5-year business case on today’s agenda. Nobody’s tested the idea, and the sponsor wants a yes or no by lunch.

What the spreadsheet’s resting on:

• A guess about whether customers will change how they work.
• An adoption curve borrowed from another product.
• Cost estimates from a team that’s never built this.
• A payback date nobody in the room can check for 3 years.

A yes commits the full sum to a forecast. A no kills an idea that might have worked, on the same thin evidence.

So approve neither, and fund your next test instead: a small experiment with one question, a capped spend, and a decision rule set in advance.

Then your board stops asking “will this work?” and asks how to test it cheaply. Each answer funds or stops the next test, and big bets still come later.

Why a Business Case Can’t Carry an Uncertain Idea

Dan Lovallo and Daniel Kahneman described the problem in Delusions of Success.

Biases and organizational pressure, they wrote, lead managers to “make overly optimistic forecasts” on major investments.

And optimism isn’t the only thing working against the forecast:

• Targets dressed as forecasts. Kahneman, Lovallo, and Olivier Sibony draw the line in a 2011 HBR article: “Forecasts should be accurate, whereas targets should be ambitious.” A funding request can blur that line, because a sponsor who needs an ambitious target to win the slot can end up presenting it as a forecast.
• Averages that hide the tail. In a 2011 study of 1,471 IT projects, Bent Flyvbjerg and Alexander Budzier found the average cost overrun was 27%. Yet one in six was a black swan, with cost overruns averaging 200% and schedule overruns of almost 70%. They’re IT projects, so treat them as an analogy.
• New-venture probabilities that can’t be known up front. In Entrepreneurship as Experimentation, Kerr, Nanda, and Rhodes-Kropf call success probabilities “low, extremely skewed, and unknowable until an investment is made.”

Lovallo and Kahneman’s fix is the outside view, also called reference-class forecasting. You study a class of similar projects and place yours inside that distribution.

But a genuinely new idea may have no reference class that’s worth the name. So the forecast falls back on the team’s own story about itself.

In practice, that story’s a stack of assumptions. Each one looks reasonable, and none has been tested yet.

That leaves the board grading a story, and a confident presenter gets an edge the evidence hasn’t earned.

A business case is still the right tool later, though. Once the big unknowns are settled, it’s how you size and commit the full investment.

What a Big Yes Does to the Board Afterward

Approval can change the people who gave it, and 2 studies point to how:

• Personal responsibility. Barry Staw tested escalating commitment in a 1976 lab experiment, Knee-Deep in the Big Muddy. His participants “committed the greatest amount of resources to a previously chosen course of action when they were personally responsible for negative consequences.” It’s a lab finding, so read it as a direction.
• Group belief. Isabelle Royer’s HBR article describes “a fervent belief in the inevitability of a project’s ultimate success.” That belief spreads until signs of doom look like “temporary setbacks.” Her opening case, RCA’s SelectaVision, “had cost the company an astounding $580 million and had tied up resources for 14 years.”

Here’s our read: everyone who voted yes now owns the whole bet. A missed milestone feels like an attack on their judgment, and the next review defends the vote.

Royer’s remedies are practical: teams “not entirely composed of like-minded people,” “well-defined review processes” that stick, and an exit champion who challenges the team’s assumptions.

Look at what those remedies share. Each one builds disagreement into the process, ideally before anyone’s attached to the outcome.

Gate frameworks already offer a way out. The Stage-Gate decision menu lists “Go (fund), Kill (stop funds), Recycle (obtain information), Hold (fund pending resource availability).”

Boards that only use Go and Kill skip the option built for uncertainty. Recycle, with a written question attached, is the experiment approval.

The Arithmetic That Favors the Experiment

Kerr, Nanda, and Rhodes-Kropf work through an example: a project that costs $110 to commercialize and pays $10,000 with 1% probability, $0 otherwise.

The numbers are toy-sized on purpose. Multiply them by 10,000 and the arithmetic holds at board scale.

Funding now is worth minus $10. The test comes back positive 1 time in 10, and a positive result lifts the odds to 10%, worth $890.

So the authors run the test “as long as the experiment costs less than $89” (10% x $890), then shut down or commercialize the project on the result.

Real experiment data points the same way. Ron Kohavi and Stefan Thomke’s 2017 account of online experiments covers Microsoft, Bing, and Google.

At Google and Bing, only about 10% to 20% of experiments generated positive results. Even experts, they write, “often misjudge which ones will pay off.”

And big builds miss too: Bing’s social-media integration cost Microsoft more than $25 million for negligible gains. Testing a low-priority idea for displaying ad headlines lifted revenue 12%.

What the Board Should Approve Instead

The approval should cover one test, written down before the work starts. That experiment approval fits on one page:

Start with the question row. Intuit describes leap-of-faith assumptions as those “most crucial for a product or feature to succeed.”

They also “have not yet proven to be true in practice elsewhere.” If an assumption has already held somewhere comparable, test the next one.

For the measure, the Seven Pitfalls paper Kohavi co-authored defines the Overall Evaluation Criterion as “A quantitative measure of the experiment’s objective.”

It warns against one “for which it is easy to beat the control by doing something clearly ‘wrong’ from a business perspective.”

The last row borrows from venture capital, as Kerr’s paper describes it.

Investors “tie each financial infusion to milestones,” sizing each round to the specific uncertainty it resolves.

A Worked Example From Operations

Here’s an illustrative case. An operations team proposes predictive maintenance and asks for EUR 900k to roll it out across 8 sites.

The model has worked elsewhere, so the riskiest assumption is human. The board approves this, in writing:

• Question: will site technicians act on the alerts?
• Measure: the share of alerts that led to a logged work order within 48 hours.
• Cap: EUR 20k and a 6-week pilot at one site.
• Decision rule: continue at 60% or more, stop below 30%, and change anything in between.
• Date and decider: the board meeting in week 7, chaired by the innovation director, who has the final call.
• Next tranche: continue funding 3 more sites up to EUR 300k; change repeats the EUR 20k pilot (a 2nd middle result stops); stop closes it.

Every number above was set before the pilot started. If the share comes back at 45%, nobody argues about what it means.

It means change: maybe the alert wording, maybe the shift that receives it. The board funds one more round and keeps the cap.

At 20%, the stop is already agreed, and the rest of the EUR 900k stays unspent.

Kerr and his co-authors note that “an important role of venture capital firms is to shut down ventures that are receiving bad signals.”

The cap follows the logic of Kerr’s toy example. EUR 20k is small against what a clear answer is worth to a EUR 900k decision.

How the Board Decides Together When Results Come Back

When results arrive, hindsight starts rewriting what everyone expected. Brian Nosek and colleagues point to “hindsight bias” in The Preregistration Revolution.

It makes predictions and postdictions hard to tell apart. Their fix: “define the research questions and analysis plan before observing the research outcomes.”

The experiment approval does that job for a board. Then the review itself needs a fixed order:

1. Each member records continue, change, or stop against the decision rule, alone.
2. The chair reads every call aloud, splits included.
3. The board discusses the gaps between the calls and the rule.
4. The named decider makes the call, and any override of the decision rule gets logged with its reason.

Step 1 borrows from Gary Klein’s project premortem, where participants “independently write down every reason they can think of for the failure.”

Step 1 also blunts the HiPPO, the “Highest Paid Person’s Opinion,” as defined in a 2013 paper Kohavi co-authored. Senior members write their call down before anyone talks.

Step 2 exposes fake agreement. Kahneman, Lovallo, and Sibony write that “an absence of dissent” on a complex problem “should sound an alarm.”

For step 3, software can help. An AI summary of the evidence against the decision rule, checked by the chair, gives every member the same picture.

Step 4 is our firm rule: an override is allowed, but the reason goes on record. Quietly moving the goalposts after the results is the failure mode.

When a Big Bet Is the Right Call

Experiments aren’t always the right call. In his 2016 letter, Jeff Bezos wrote that reversible “two-way doors” “can use a light-weight process.”

His 2015 letter reserved slow, careful process for one-way doors, the calls you can’t undo. Here are signs that the idea needs a big-bet decision instead:

• The decision can’t be reversed once it’s made.
• Nothing meaningful can be learned below full scale.
• The test itself would harm customers or the brand.
• Delay costs more than being wrong.

The 2nd sign mirrors the Kerr caveat. Experiments pay when early information is “especially informative” about the project and “cost effective to obtain.”

The 3rd comes from Stefan Thomke’s work at Bank of America, which ran service experiments in live branches. He warns that “live tests magnify the cost of failure.”

For the 4th, Bezos’s rule of thumb applies: decide with “somewhere around 70% of the information you wish you had.”

Small experiments have a limit too. Kohavi’s 2013 paper warns that “you don’t get to the moon by climbing higher and higher trees.”

Even a big bet deserves staging, plus the outside view where a reference class exists. Fund the irreversible part last, and test what’s still testable at small scale.

Fund the Next Test, Then Decide

When nobody can honestly say whether an idea will work, the business case can’t decide it. Your board can still fund the next test.

Try these at your next board meeting:

• Ask each sponsor to name the one assumption that would sink the idea.
• Send back any funding ask that lacks a decision rule written in advance.
• Have every member record a call alone before discussion starts.
• Check last year’s approvals and count how many came back with a clear stop, change, or continue.

Expect slower first reviews and some sponsor pushback on the cap. But remember that a capped round lets the board say yes sooner than a full business case would.

And a stop closes only the version that was tested, so a sponsor with a new question can bring it back to the board.

Over time, each approval buys an answer, and your board stops defending bets it can’t check. So fund your next test, and let its answer decide the one after.

Download our ebook on idea evaluation and governance. It covers structured pipelines, scoring methodologies, governance frameworks, and the handoff from approved ideas into portfolio management.

Or book a demo to see how Accept Mission handles scorecards, decision workflows, and audit trails, so every approved experiment has a visible outcome.