Measuring learning progress: using learning analytics well
Short answer
Learning progress is best measured not by clicks or time spent but by whether learners answer questions correctly, solve tasks and can apply their knowledge. Sensible learning analytics combines completion data with knowledge levels per topic and shows learners themselves where they stand; for organisations, aggregated figures are usually enough.
Key points
- Completion is not understanding: a course clicked through proves little.
- Answers to questions, exercises and application tasks are what carry meaning.
- Knowledge level per topic shows gaps more precisely than an overall score.
- Learning analytics serves the learner first, then the organisation.
- Data protection and co-determination set clear limits for personal analytics.
What can be measured — and what counts
Learning platforms record a lot of data: logins, pages opened, time spent, video minutes watched, clicks, completions, test results. Most of it measures activity, not learning. Letting a video run to the end does not mean you understood it. Clicking quickly through a course may mean you knew the material beforehand.
More meaningful is data that shows whether someone can do something: answers to comprehension questions, results in exercises, explanations in their own words, solved practical tasks. This data takes more effort to collect, but only it answers the real question.
| Metric | What it shows | Limit |
|---|---|---|
| Completion rate | How many finished the course | Says nothing about understanding |
| Time spent | How long someone was in the course | Long can mean interest or confusion |
| Test score | How many questions were right | Only as good as the questions |
| Knowledge per topic | Where someone is confident and where not | Needs enough answers per topic |
| Application task | Whether knowledge is used in a situation | Effortful to assess |
| Impact at work | Whether behaviour or results change | Hard to attribute clearly |
From completion to knowledge level
A knowledge level groups answers by topic and shows, for each topic, how confident someone is. Instead of “course passed with 75 percent” you see: basic terms secure, application to cases still shaky, one specific misconception still open. That is more useful to learners than an overall score because it suggests a next step.
For a knowledge level to be reliable, it needs enough answers per topic and questions that genuinely test understanding. A platform that asks a question after each slide, follows up on gaps and detects misconceptions comes closer to a realistic picture than a single final exam.
Measuring impact beyond the course
In workplace learning, a four-level model for evaluating training has become established: reaction (How did participants like the training?), learning (What did they learn?), behaviour (Do they apply it?) and results (What did it bring the organisation?). Learning platforms cover the first two levels well. The third and fourth require data from everyday work, such as error rates, customer satisfaction or feedback from managers.
Be honest about attribution. If fewer errors occur after a training course, that may be due to the training, but also to other changes. Comparing with a group that is trained later makes the conclusion more robust.
Data protection and trust
Learning analytics works with personal data and, in companies, touches on co-determination. A good rule is: detailed learning data belongs to the learner; organisations see aggregated figures and the records they genuinely need. Anyone who has to fear that every mistake in an exercise will end up in their personnel file learns cautiously rather than honestly.
Define who sees which analyses, how long data is kept and what it may not be used for. Put this in the privacy notice and, where one exists, in the works agreement.
Learning analytics for learners
The greatest value of learning analytics lies with learners themselves. An overview that shows where you are confident, where you should do more and whether you are ready for the exam helps more than any HR report. Good platforms show such overviews directly and suggest suitable next steps: a review, an exercise, an example.
Honesty in presentation matters. A percentage with one decimal place suggests a precision that does not exist when there are only a few answers. Rough levels with a reason are better, such as “secure”, “still uncertain” or “not yet assessed”, together with a note on what the assessment is based on. Then learners understand what they can do to improve it.
How Universe does it
In Universe, the knowledge level per course shows exam readiness, strong areas, areas to deepen and open misconceptions; slides, quiz answers, questions and detected misconceptions feed in continuously. For organisations there are training records and a manager view limited to the team. Learning events can be sent via xAPI to an external learning record store.
Terms
- Learning analytics
- Analysis of learning data to make learning visible and improve it.
- Knowledge level
- An estimate of how confident someone is in individual topics of a course.
- Misconception
- A wrong understanding of something that shows up in answers.
Frequently asked questions
What is learning analytics?
Learning analytics is the collection, analysis and presentation of learning data in order to improve learning and learning environments.
Is completion rate a good metric?
It is useful for compliance training because it shows whether deadlines are met. It says little about learning success. Add test results and knowledge levels per topic.
May managers see individual employees’ learning results?
That depends on the legal basis, purpose and, where applicable, a works agreement. Typically managers see completion and deadlines, but not details from exercises.
How do I measure whether training works on the job?
With data from everyday work, such as error rates or feedback, ideally compared with a group not yet trained. Learning platforms alone cannot measure this.
How often should learning data be analysed?
Continuously for compliance training, so that deadlines are not missed. For course quality, a quarterly review is usually enough: where do learners drop out, which questions are answered wrongly unusually often, which slides trigger many follow-up questions?
How it works in Universe
Related articles
- Adaptive learning paths: learning that adjustsHow adaptive learning paths work: assess the starting point, spot gaps, choose next steps, check again — and where AI helps.
- Writing exam questions: multiple choice and open-endedHow to write fair exam questions: good multiple-choice items with plausible distractors, open questions with a marking rubric, and AI as an assistant.
- LMS for companies: what matters in corporate learningWhat a corporate LMS must deliver: onboarding, compliance training, records, a manager view, works council rules and integration with HR systems.
- SCORM, LTI, xAPI and cmi5 explained simplyThe key e-learning standards compared: what SCORM, xAPI, cmi5, LTI, QTI and Common Cartridge do and which standard fits which job.
- Writing learning objectives with Bloom’s taxonomyHow to write testable learning objectives: the six levels of the taxonomy, suitable verbs, examples, common mistakes and the link to assessment.