Your LMS knows who completed what course and when. That is roughly where its usefulness as an evidence source ends. If you’re trying to answer “did the training work?” — did behaviour change, did quality improve, did the spend pay for itself — the LMS on its own cannot tell you, and treating its reports as measurement is one of the most common and most expensive habits in corporate learning. This article sets out exactly what LMS data can support, what it can’t, and what to connect it to so your training investment decisions stop resting on completion counts.
I run Learning Elements and the Training Intelligence Model (TIM), and I look at training data across organisations for a living. The pattern is consistent: the LMS is a logistics system, and its reports get promoted into evidence they were never designed to be.
What LMS data is actually good for
Give the LMS its due first. It is genuinely good at four jobs:
- Compliance tracking: who has completed (or not completed) mandatory training, by when, with a timestamped record you can produce in an audit.
- Logistics: enrolments, scheduling, seat utilisation, spend per course.
- Catalogue economics: which courses are actually used, and what each seat costs.
- Level 1–2 signals: survey responses and quiz scores — useful for course-quality housekeeping, not for impact claims.
Those are real functions. The failure isn’t the LMS; it’s the moment a completions report is labelled “training effectiveness”.
The five things your LMS cannot tell you
1. Whether anyone changed how they work. Completion proves exposure, not behaviour. A 100% completion rate on a difficult-conversations module says nothing about whether difficult conversations are now handled differently on the floor. Behaviour evidence lives in operational systems: QA scores, call recordings, CRM notes, error logs, supervisor observation.
2. Whether anything improved as a result. Outcome data — quality, productivity, safety incidents, customer measures, attrition — sits in operational and HR systems, keyed to people and teams, not to course completions. Without joining those records to training records, you cannot see a relationship at all. The LMS has no reason to hold that join, and mostly doesn’t.
3. Whether the learners who “passed” can apply anything. Multiple-choice completion quizzes measure recall minutes after exposure. They don’t measure transfer to real work, and a pass mark tuned for accessibility is rarely tuned for discrimination between competent and not-yet-competent.
4. Who actually engaged. An “in progress” status can mean focused study or a video running in a background tab. Time-in-content recorded by the platform is a weak proxy at best, and it degrades further whenever content is consumed off-platform.
5. What the training cost in full. LMS spend reports capture licence and delivery costs. They don’t capture the largest line item — participants’ time — let alone manager time, backfill, or travel. Any ROI calculation built on LMS cost data alone understates cost, often by a factor of two or more.
The completion-rate trap, in numbers
Consider what a typical LMS dashboard reports versus what a decision-maker actually needs:
| Dashboard metric | What it proves | What it doesn’t prove |
|---|---|---|
| Completion rate: 94% | Records were marked complete | Learning, behaviour, or outcome change |
| Average quiz score: 87% | Short-term recall of material | Application on the job |
| Hours consumed: 3,200 | Time was logged | Attention, quality, or transfer |
| Satisfaction: 4.4/5 | Participants liked it | Anything downstream of the room |
None of these metrics is useless. Together, as a picture of “effectiveness”, they are close to empty — and they are what most training budget reviews are defended with.
The structural reason: the LMS is an island
The deeper issue is architectural. Evidence that training worked lives in at least four different systems:
| Evidence type | Where it lives | Example signal |
|---|---|---|
| Exposure | LMS | Completion records, dates |
| Behaviour | Operational systems | QA scores, error rates, observed practice |
| Outcomes | BI / HR / finance | Quality, attrition, incident rates, revenue per FTE |
| Context | HRIS, rosters, org charts | Tenure, team, role, concurrent changes |
Training data quality is determined by the joins, not the records: a completion record joined to an outcome measure is evidence; either alone is bookkeeping.
That definition is the whole game. Organisations with genuinely useful training measurement have built the joins — person-level and team-level links between exposure and outcome records. Organisations without them keep buying better dashboards on the same unjoined records, which changes the presentation and nothing else.
A five-step fix you can start this quarter
- 1. Pick one programme that matters. Not the whole catalogue — the one programme where the money and the stakes are biggest. Measurement credibility is built one well-measured programme at a time.
- 2. Name the behaviour and find its existing signal. Before any new tooling, establish what operational record already changes when people do the job differently. If none exists, that finding reshapes the programme itself.
- 3. Establish the baseline now. Pull 6–12 months of the target signal, split by team where possible. You cannot retroactively create a clean baseline, so this step is urgent even before anything is decided.
- 4. Stagger the rollout if you can. Two cohorts a month apart gives you a comparison group almost for free — the single cheapest improvement in attribution you can make.
- 5. Build the join, even in a spreadsheet. Export LMS completions, export the operational signal, join on person/team, and look at before/after. A defensible spreadsheet beats an unjoinable dashboard.
The honest checkpoint: run that join for your biggest programme. If you can’t produce a single row that says “these people trained, and here is what their work looked like before and after”, your organisation has completion tracking, not training measurement. That gap is exactly what the Training Intelligence Model (TIM) is built to close — connecting exposure, behaviour, outcome and context data into evidence that can survive a budget review or an audit. For the financial side of that evidence, see the complete guide to measuring training ROI, and for how completion records become audit-defensible evidence, see training compliance risk: from completion records to audit-ready evidence.
No report solves it, because the limitation is the data model, not the presentation layer. LMS records are keyed to users and courses; outcome records are keyed to people, teams and time. Until those records are joined, no amount of visualisation turns completions into evidence.
Usually no — at least not first. Start with one programme, existing operational signals, and a spreadsheet join. Platforms become useful once you know exactly which data you need joined and why; buying one before that tends to automate the same unanswered questions at higher cost.
xAPI (and Caliper) let learning systems emit richer event streams, and they help with the exposure and in-course-engagement layer. They don’t manufacture behaviour or outcome data — those still have to come from operational systems and be joined deliberately. Treat xAPI as one input, not a measurement strategy.
Ask for a full person-level export of completion and assessment records, including timestamps — not a dashboard screenshot. Then ask whether their reporting can join to data you supply. If both answers are good, your LMS is fine; the gap is in what you feed it.
If you want an external, structured look at what your current training data could and couldn’t support, that’s the conversation TIM exists for. Start it here, or connect with Peter on LinkedIn.
