Quick answer: Analyze pre-post data person by person, not average by average: compute each respondent's delta, test it with a paired method (Welch's t-test), size it with Cohen's d, and disclose the traps — regression to the mean, floor/ceiling effects, and dropouts — before leadership finds them for you.
The design guide covered building the instrument. This is the harder half: you have two columns of scores and a leadership meeting on Thursday. What can you honestly claim?
The Three Questions Your Data Must Answer
- Did scores go up? — direction, per person
- By how much? — magnitude, in the units your stakeholders care about
- Is it real or noise? — statistical confidence, stated at the right humility for your sample size
Teams that answer only the first question get challenged on the second in the meeting. Teams that answer all three, including the caveats, get funding.
Paired Analysis: Compare People to Themselves
The Traps That Fake Improvement
People who scored unusually low on the pre-test improve on average even if you teach them nothing. If your headline is "the bottom quartile improved 31%," most of that is statistics, not pedagogy.
A pre-test everyone aces shows zero gain; one everyone fails shows gain from exposure alone. Check the score distributions before claiming the instrument measured anything.
"Respondents who completed both measurements (n=38) improved on average +11.4 points (Welch's t-test, p < 0.01, Cohen's d = 0.72). Gains concentrated in the mid band; high scorers were near ceiling. Dropouts (n=6) skewed toward the operations role — noted, not silently excluded."
The Group Report Leadership Actually Reads
One page, in this order: the headline delta with its confidence statement; the distribution chart (how many moved, how far); one segment cut that matches the audience's obsession (by department, by role, by region); and the caveats box. The caveats box is not humility theater — it's the section that makes the other three credible.
Building this by hand from two CSV exports is where most teams quietly stop measuring. An assessment platform that scores each response and aggregates the paired deltas turns the report into a download — the value-report workflow covers the full setup, and FormLM's insight reports compute the Welch t-test and Cohen's d per group automatically.
The goal of pre-post analysis is not to prove the training worked. It's to produce a number leadership trusts enough to ask "so what do we do differently next quarter?" — which is the question that keeps training budgets alive.
✅ Key Takeaways
- Analyze per-person deltas, never two cohort averages
- Show the distribution — a mean hides who actually moved
- Pair the significance test with an effect size; each alone misleads
- Disclose regression to the mean, floor/ceiling, and dropouts before someone else does
- One-page report: headline, distribution, one relevant segment, caveats box
Skip the Spreadsheet Merge
Collect pre and post rounds in one assessment, and get the paired deltas, significance, and group report generated per cohort — exportable as a branded PDF.
Try FormLM →Frequently Asked Questions
What statistics should I use for pre-post test data?
Compare each person's own before and after scores (paired analysis), not two group averages. Report the mean change plus a paired significance test such as Welch's t-test, and an effect size like Cohen's d — significance says the change is real, effect size says whether it matters.
Why do low scorers always seem to improve the most?
Mostly regression to the mean: anyone who scored unusually low on the pre-test tends to score closer to their true level on the post-test even with no training at all. Segment results by pre-test band and keep the interpretation honest about this effect.
How many responses do I need for a valid pre-post analysis?
Paired designs are efficient: around 20-30 matched pairs can detect a moderate effect. Below that, report the deltas descriptively — mean change and distribution — without claiming statistical significance.