Use an ability score to design the training, not to label the person

A cement plant in Gujarat ran its supervisors through an ability test in March, as the first step of a leadership programme. The HR team wanted to know who would benefit most. By June the programme had lost nine of its twenty two participants, and the exit conversations kept producing the same sentence: I am not the academic type.

Nobody had told them they were below average. They had been shown a report with a percentile, a shaded bar, and the word "potential" at the top.

The test was not the problem. Turning its result into a statement about a person was. That turn is avoidable, and it costs about an hour of design work before the test is ever sat. It is also a different task from scoring assessment reports, which is where most teams spend the budget.

An ability score answers a narrower question than the one being asked

A cognitive ability test measures how well someone handles novel problems under time pressure, with unfamiliar material and nothing to look things up in. That is a decent proxy for one thing: how quickly a person absorbs new content when it is presented once and applied soon after.

It says nothing about how far that person can go in a job. Performance also depends on what they already know, how much they care, whether the manager removes obstacles, and whether the work suits them. The test is blind to all of that by design. It was built to rank candidates against each other, not to plan what to teach them.

The published numbers say this cleanly. In a meta-analysis of twenty years of training research, Colquitt, LePine and Noe found cognitive ability correlated .69 with declarative knowledge gained in training, .38 with skill acquisition, and .43 with transfer back to the job. All three are respectable. The drop from .69 to .43 is the whole story for development work.

Ability forecasts how much someone will understand by Friday, strongly. It forecasts whether they will do anything differently next month, moderately. Use one number to plan both and you get quiz scores with no behaviour change, then you blame the participants. It is the same misreading as treating a percentile as a fact about a person, which a norm group cannot support.

What the score can decide, and what it cannot

Write this down before anyone sits the test.

The left column is design. The right column is judgement. Nearly every demotivation story in this category is a design input that crossed into the right column, usually because handing the score to a manager was easier than rewriting a module. A promotion slate built on test scores selects for test performance.

Build the band before anyone sees a number

If a score is going to change a design decision, the smallest defensible unit is a band, not a point. The Standards for Educational and Psychological Testing is explicit that the standard error of measurement should be used to put confidence intervals around a reported score, and that conditional standard errors at particular score levels are often larger than the single figure for the whole group.

A test with a standard deviation of 15 and a reliability of .90 has a standard error of 15 times the square root of .10, about 4.7 points. The 95 percent band is roughly plus or minus 9. Two people who score 96 and 104 are indistinguishable on that instrument. Splitting them into tiers is rounding, not evidence.

Reliability of .90 is a good test. Short online instruments often sit nearer .80, where the band widens to plus or minus 13 on the same scale. At that width you can honestly report three bands. Not five, and not a percentile to the whole number.

The band rule also ends the argument about who "just missed" a threshold. They did not miss it. The instrument cannot see the difference.

Match the band to structure, and only to structure

This is the one place the score earns its keep, and the evidence is older than most vendors selling into it.

Cronbach and Snow's work on aptitude-treatment interaction is usually summarised badly. Snow's own later account is that interactions are common, most are hard to demonstrate, and none is understood well enough to build a whole method on. But one pattern survived well enough to act on: treatments that differ in structure and completeness interact with general ability. Highly structured instruction, with defined sequences, worked examples and external control, tends to help lower-ability learners and can hinder higher-ability ones, who do better with less scaffolding.

As a design decision:

Nothing in that table says who attends, who gets promoted, or who is a high potential. It says how to write the workbook.

The limit sits in the same source. If you cannot rewrite the material, you cannot use the score this way, and then you should not test at all. Testing without a design lever is data collection that produces a label and nothing else.

A worked example, with illustrative numbers

Twenty eight service engineers in a two-state territory are booked onto a six-week diagnostics programme: four modules, a fixed workbook, everyone attends.

Step one, before testing. The training manager lists the levers available: number of worked examples, number of supervised practice attempts. Two levers. That constrains what the test is allowed to decide.

Step two. The test returns a reliability of .88 and a standard deviation of 15. Standard error is 15 times the square root of .12, about 5.2. The 95 percent band is plus or minus 10, so the programme reports three bands, not a rank order.

Step three. Trainers, not statistics, set the band cut. Past cohorts show that people needing more than three supervised attempts on module one rarely finish in six weeks. That is a design fact, and it anchors the cut.

Step four. The workbook is rewritten for the lower band only: two worked examples per module, supervised attempts doubled from two to four. Cost, about eleven trainer hours. It covers nine of the twenty eight.

Step five. Outcomes are read from two parallel forms of a knowledge test plus a supervisor observation on the floor, not from the ability test again. If the lower band's knowledge gains close the gap and the observation scores move, the design change worked. If only the knowledge scores move, you taught them the workbook, not the job.

Those numbers are illustrative. The sequence is not. Four of the five steps happen before any participant sees anything.

The debrief is where the score usually does damage

Kluger and DeNisi pooled 607 effect sizes from 23,663 observations and found feedback improved performance by an average of d = .41. Over 38 percent of those interventions decreased performance. Their explanation: feedback shifts attention, and the ones that go wrong pull attention away from the task and toward the self.

Mueller and Dweck found the matching pattern. Fifth graders praised for intelligence rather than effort preferred performance goals over learning goals, and after a setback showed less persistence, less enjoyment, more low-ability attributions and worse performance. Praise for ability, given after a success, taught them that performance reveals intelligence. So failure later meant something about them.

Adults are not fifth graders and a percentile is not praise, but the mechanism is the same. A debrief that says "your reasoning score puts you in the third quartile" hands someone a fact about themselves and no plan. A debrief that survives a sceptical supervisor runs in this order:

1. What the test measured, in one sentence, including what it did not measure.
2. Which band they are in, with the width stated out loud so the number cannot harden.
3. What the band changes in their programme: examples, attempts, review sessions. Named and dated.
4. What the score does not predict. It does not tell us how you will perform on the floor, and we will judge that from the floor.
5. What happens next, and who they speak to if they think the result is wrong.

Where ability data genuinely helps a development plan

Pacing. If a module will take some learners twice as long, plan the schedule and the trainer hours instead of discovering it in week three. Boring, and the most valuable use on the list.

Structure, as above. The one design lever with a defensible research base.

Knowing when a test is the wrong tool. A meta-analysis of 8,776 effect sizes found the correlation between pretest and posttest knowledge was .534, while prior knowledge against normalised gain was -.059, with a 95 percent prediction interval from -.688 to .621. Where someone starts is fairly stable. How much they gain is nearly unrelated to it. If your question is what this person already knows, test that directly.

What is missing from the list is prediction of individual success. A .69 correlation at group level says nothing reliable about one person in one cohort, and the band width is the arithmetic proof.

The honest limit

The structure-matching rule rests on interaction research that Snow called hard to demonstrate and too poorly understood to build a method on. It gives a direction, not a curriculum. If the programme is short, the cohort small and the material fixed, skip the ability test and spend the money on practice attempts for everyone. What you would learn is closer to what a two-form pre and post test tells you anyway, and that one does not follow the participant into a performance review.

There is a political limit too. In most Indian organisations a cognitive test in a development context reads as a selection test in a different shirt, and participants are often right, because the score lands in the same HR file. If you cannot guarantee that separation in writing before the test is announced, do not run it.

Before the next cohort

Take the last ability report your organisation produced for a development programme and answer one question: what did it change? If the answer is a list of names and a set of tiers, the test labelled people and nothing more, and those labels are now part of how they describe themselves.

Then write the two-column table for the next programme, name the one design lever the score is allowed to pull, and fix the band width before the first score arrives. If the left column comes out empty, cancel the test and put the fee into trainer hours.