Session

The successful pilot that did not pass the investment test

Testing attribution before an attractive pilot becomes an enterprise commitment

PAWorld27 Zurich · 3–4 March 2027 · Zürich

← Back to agenda
– · Main room

About this session

A pilot team reports stronger output after receiving new AI tools. In this worked scenario, an evaluation reconstructs the baseline and finds that demand, staffing and case allocation also changed. A phased comparison and sensitivity tests reduce confidence in the claimed effect.

The session examines how to design that evaluation, and how to tell insufficient evidence of value from evidence that a tool cannot work, so that the next step is a smaller, better-targeted test rather than a full licence expansion or an outright rejection.

Learning outcomes

  • Identify the comparator and confounders behind a before-and-after claim.
  • Specify a feasible phased evaluation and its remaining limitations.
  • Use a minimum worthwhile effect and an uncertainty assessment to fund, defer or stop.

Strategic questions

How much uncertainty is acceptable for a reversible commitment compared with a costly one?

Context

Business cases often rely on before-and-after changes that coincide with shifts in staffing, demand or allocation. Investment decisions need to account for what would have happened anyway and for how much uncertainty matters to the commitment being made.

Who should attend

People analytics leaders and practitioners; workforce planning, skills and HR technology colleagues; and the business, Finance and Operations leaders who own the decision.

Participation

Built around a worked scenario rather than one organisation's reported results. The evidence, methods, assumptions and limits are shown in full, followed by chaired questions.