Which AI Use Cases Are Worth Piloting? A Risk and Value Framework for Funding, Scoping and Scaling AI
Which AI Use Cases Deserve a Pilot? A Risk and Value Framework for Funding, Scoping, and Scaling AI
Translation in progress — untranslated text is omitted.
The framework uses an AI use case with a proposed scope as the decision unit; expansion is authorized only if residual risk is acceptable and the evidence of incremental net value applies to that scope. The risk track is an R1-R4 decision table with veto items, and the value track has four evidence stages; in the synthetic scenario, the highest-value control design failed the expected-loss test, while a smaller scope passed. A Python package, with 279 passing tests, is demonstrated on synthetic refund, reconciliation and knowledge assistant use cases; the thresholds are prototype policies requiring field validation.
- Composite scores can rank AI use cases but do not state the permitted operating scope or the evidence required for expansion. The authors argue that the decision unit should be "use case × proposed operating scope", since different scopes have different risks and evidence needs.
- Proposes the PilotGate framework: three decisions (screen the scope, fund experiments, authorize expansion), an R1–R4 risk decision table with vetoes and priorities, a V1–V4/E1–E4 value evidence ladder where each transition requires independent evidence, funding using an EVPI upper bound and EVSI, expansion with a quantified expected-loss test, and controls scored only on measured evidence applicable to the scope.
- All demonstrations use synthetic data: RA passes both gates and expands (annual net value $465,234), RB fails the risk gate and requires redesign, and RC is permitted only for shadow running; in synthetic scenario RF, the design with the highest net value, D2, does not pass the expected-loss test; the software package passes 279 tests.
- The authors state that the risk table and thresholds are prototype policy and require field validation; the transfer assumptions are based on only eight weeks of observation; all scenario parameters are synthetic; the October 2026 R-BED version could not be obtained; the framework does not constitute a compliance determination.