Most training dashboards are full of numbers that feel like evidence but predict nothing: completion rates, satisfaction scores, quiz passes. They measure whether training happened. They say almost nothing about whether it worked. This article separates the metrics that merely record activity from the ones that actually predict return on investment.
Which training metrics actually predict ROI?
Only two families of metrics predict ROI: behaviour change (did people do their work differently?) and business outcome movement (did the metric the training targeted move, net of everything else?). Everything else — completions, satisfaction, attendance, even assessment scores — is an input measure. Useful for operations, worthless for the ROI case.
The four levels, and where the money is
The Kirkpatrick model remains the common language of training evaluation, and the Phillips ROI Methodology extends it with a fifth level. Understanding where each level sits on the money map explains why most organisations stall:
- Level 1 — Reaction. Did participants find it useful? Cheap to collect, near-zero predictive value for ROI. A happy cohort can learn nothing; a uncomfortable one can learn a lot.
- Level 2 — Learning. Did knowledge or skill change? Assessment scores, certifications, observed skill demonstrations. Necessary but not sufficient — people can pass tests and still not change how they work.
- Level 3 — Behaviour. Are they doing the job differently on the floor? This is the first level with real predictive power for ROI, and the first that requires data from outside the learning system: observation, manager input, work samples, system-of-engagement data.
- Level 4 — Results. Did the business metric move? Conversion rate, error rate, resolution time, safety incidents. This is where ROI lives.
- Level 5 — ROI (Phillips). The financial translation: benefits converted to money, divided by fully loaded cost. Level 5 is arithmetic on top of Level 4 — if Level 4 is missing, Level 5 is fiction.
The pattern is clear: predictive power lives entirely in Levels 3 and 4, and both require data the LMS does not have. That is the structural reason ROI measurement fails in organisations that try to do it from the learning platform alone.
The metrics hierarchy, ranked by predictive value
A practical ranking for any L&D leader building a scorecard:
- 1. Business outcome delta — the change in the target metric, isolated. Highest value. Hardest to produce without joined data.
- 2. Behaviour change rate — the proportion of trained staff demonstrably doing the new behaviour 30–90 days post-programme. The best leading indicator of ROI.
- 3. Competency sustainment — how many people are still at standard a quarter later (not just at certification).
- 4. Learning gain — pre/post assessment improvement. A hygiene metric; it gates the rest but predicts little on its own.
- 5. Completion and satisfaction — logistics and early-warning signals only. Report them if you must, but never let them carry the value story.
One quotable rule: a completion rate is a record of delivery; a behaviour change is a record of impact; only one of them belongs in a board paper.
Why behaviour data is the best early ROI signal
Business outcomes take time — a quarter or more of noise before signal emerges. Behaviour change shows up faster and earlier. If 40% of trained agents are demonstrably using the new objection-handling structure within a month, and those agents’ conversion rates are diverging from the untrained cohort, you have a directional read on ROI months before the financial number is computable.
The catch: behaviour data rarely lives anywhere systematic. It hides in QA scorecards, call samples, manager observations and CRM activity logs. Organisations that can join those sources with learning data get an early-warning system for training value. Organisations that cannot are flying blind until the outcome data arrives — which is the gap the Training Intelligence Model is designed to close: joining learning data with operational, behavioural and sentiment data so Levels 3 and 4 are continuously visible rather than annually reconstructed.
A worked example: separating signal from theatre
Two programmes, same $60,000 cost:
- Programme A: 95% completion, 4.6/5 satisfaction, 88% assessment pass. No behaviour measurement, no outcome linkage. On paper: excellent. On evidence of value: none.
- Programme B: 78% completion, 4.1/5 satisfaction, 71% assessment pass — but 55% of trained staff visibly changed their call structure within 6 weeks, and the trained cohort’s first-call resolution moved from 68% to 74% while the untrained comparison cohort stayed flat. On paper: mixed. On evidence of value: strong.
Programme B is worth repeating. Programme A is not — despite looking better on every metric that is easy to collect. This is the trap most training dashboards are built into.
What to do on Monday
- Audit your current training dashboard: how many of its metrics sit at Levels 3 or 4? For most organisations the honest answer is zero.
- For your next programme, define one observable behaviour and one business metric before launch — and record both baselines.
- Add a 60-day behaviour check to the design: who will observe what, against what standard, with what data?
- Stop reporting satisfaction as an outcome. Report it as what it is: a delivery-quality signal.
For the financial half of this equation — converting outcome movement into a defensible ROI figure — see our companion article on how to measure training ROI.
Frequently asked questions
Are satisfaction scores completely useless?
No — they are a useful early signal on delivery quality and relevance, and very low scores are worth investigating. They are only useless when presented as evidence of impact, which they are not.
What is the Kirkpatrick model in simple terms?
A four-level framework for evaluating training: Reaction (did they like it), Learning (did they learn), Behaviour (did they change), Results (did it matter to the business). Phillips added a fifth level, ROI, which converts results into money.
How do you measure behaviour change after training?
Define the target behaviour as an observable standard before the programme, then sample it 30–90 days later using QA data, observation, work samples or system activity. Compare trained staff against an untrained comparison group wherever possible.
Which single metric should an L&D leader start with?
Behaviour change rate at 60 days. It is the earliest reliable predictor of eventual ROI and forces the organisation to define what “changed” actually means — which is the discipline most training evaluation lacks.
About the author
Peter Baumann is Commercial Director and co-founder of Learning Elements, with over 25 years across training, coaching, sales enablement and operational leadership. He works with organisations to turn learning into measurable performance, and is a developer of the Training Intelligence Model (TIM). Connect with Peter on LinkedIn.
If you run training for an organisation of 50+ people and suspect your dashboard measures delivery rather than impact, share your thoughts or discuss further with Peter.