How to build a competency framework managers will actually use

A retail chain I worked with spent four months building a competency framework. Sixty pages, nine competencies, four levels each, signed off by the leadership team in a two-hour meeting where nobody opened the appendix. Six months later I sat in a branch review and asked the manager to name three of the nine. He got one, and he was not sure about it.

The framework was not badly written. It was well written, carefully edited, and useless, because it was built as a description of people rather than as an instrument for comparing them. A framework has two jobs: help a manager decide whether this person is further along than that person, and help two managers reach the same verdict on the same behaviour. Anything that does not serve those jobs is decoration.

Why frameworks fail in the same way

Ask how the document was built. Almost always the answer is a workshop with a long list of competencies generated from job descriptions, then a consolidation exercise, then a review. Each step adds. Number of competencies grows because each function wants its own, and levels grow because someone points out that a senior manager does the competency "more". Nobody is ever asked to remove.

The document ends up optimised for coverage. The job it does requires comparability. Coverage wants a competency for every distinguishable behaviour. Comparability wants few enough that a manager can hold them in working memory while watching someone work.

There is decent evidence about how many dimensions the criterion space actually has. Bartram's Great Eight study pooled 29 validation studies covering 4,861 people and used eight competency factors as the criterion framework. Personality-based predictors correlated moderately to well with line-manager ratings on all eight. Ability tests correlated with four of the eight. Together, ability and personality data gave operational validities from 0.20 to 0.44 across the eight competencies, and 0.53 when predictors and criteria were both aggregated.

Two things follow, and they are the two things most frameworks get wrong.

First, eight factors were enough to describe the performance space across 29 studies and dozens of jobs. A framework with forty competencies is not more precise. It is measuring the same eight things at a resolution no rater can apply consistently.

Second, if ability predicts four of the eight and personality predicts all eight, then a framework with no ability dimension anywhere is a personality instrument with a business vocabulary. That is fine if you know it. It is not fine if you are using it to decide who gets promoted.

Step one: cap the list before you write anything

Do this before the workshops, because the workshops will try to undo it.

Set a hard cap of twelve competencies. Not a target, a ceiling. If a function insists on its own, it goes into the twelve or it does not exist. Then allocate: three to four that apply to everyone, four to five for people-managers, three to four for specialist tracks. A real published example of the discipline is the IAEA framework, which restricted itself to three core values, four core competencies and eleven functional competencies, and assigns an average of three to five functional competencies to any given job. That is a large international organisation with scientific, administrative and field roles. A 400-person company can manage with less.

Then test each candidate against one question: can two managers watch the same meeting and disagree about this competency in a way that a third person could adjudicate? If not, it is not a competency, it is a personality adjective. "Collaborative" fails. "Brings a decision back to the group that was taken in a side conversation" passes.

Cut "communication". It is a category, not a competency. Split it into the two things you actually mean: writing a recommendation that gets approved without a meeting, or running a review where the person being reviewed does most of the talking.

Step two: define each competency in one sentence plus a boundary

Every competency needs three pieces of text, and no more:

1. A one-sentence definition using an observable verb.
2. A boundary line stating what it is not, and which neighbouring competency that behaviour belongs to instead.
3. The work product where you would see it.

The boundary line is the piece almost every framework omits, and it is the piece that makes two managers agree. Most rating disagreement is not about how well someone did. It is about which box the behaviour belongs in. "He escalated the client problem to me" can be read as good judgement, poor ownership, or strong stakeholder management, depending on which definition you read first.

Written properly, a competency entry looks like this:

Owns the outcome. Takes a commitment through to a delivered result or a stated failure, without needing to be reminded. Not: asking for direction, which sits under Judgement, or keeping stakeholders informed, which sits under Communication. Seen in: project closure notes, the status thread on a stalled deliverable, what happens the week after a commitment slips.

That is 60 words. Nine of those fit on two pages. Sixty-page frameworks are usually sixty pages because each competency has six paragraphs of aspiration attached.

Step three: write four levels, named by scope

Four levels is the practical maximum. Five feels more precise and produces worse data, because raters cannot reliably distinguish adjacent points on a continuum they have never been trained on. Three is defensible for small companies.

The mistake is naming levels as a quality ladder: developing, proficient, advanced, expert. Nobody can tell where proficient ends and advanced begins, so every rating drifts to the middle.

Name the levels by scope of application instead, which is something a manager can observe directly:

This mapping is not invented. The IAEA framework binds its competencies to occupational roles rather than to quality adjectives: individual contributor, manager, senior manager for the core competencies, and four role levels for the functional ones. The level a person is rated at is a fact about their job, not a judgement about their worth.

If you cannot write an anchor for level 4 that a manager could see happen in a normal month, you do not need level 4.

Step four: run the retranslation test, which is the step everyone skips

This is where a framework becomes usable, and it costs one afternoon.

The method comes from Smith and Kendall's 1963 work on behaviourally anchored rating scales. They built scales by collecting real examples of effective and ineffective behaviour, sorting them into dimensions, and then asking a separate group of subject matter experts to assign each example back to a dimension without being told the intended answer. Examples that did not land back in the right place were discarded. The retention rules reported for that procedure are agreement of roughly 50 to 75 percent or better on the dimension sort, and a standard deviation below about 1.50 on the effectiveness rating.

You can run a light version with paper and a spreadsheet:

1. Print every anchor on its own strip, with the competency name and level removed.
2. Give the pile to five to seven managers, separately, in one room, with the definitions sheet.
3. Ask each to place every strip into one competency and one level.
4. Keep the strips where at least 75 percent of managers chose the same competency and no more than one level apart.
5. Rewrite or delete everything else.

In practice, expect a third to a half of your anchors to fail the first time. That is the point. An anchor that three of your seven managers file under the wrong competency will produce a rating argument in every review cycle for the next three years, and you found it for the price of one afternoon.

The retranslation also tells you which competencies are actually distinct. If every anchor for "Drives results" lands under "Owns the outcome", you have one competency, not two, and the second one is a duplicate that will split your data and your arguments.

If you want external cover for the calibration effort, note that frame-of-reference training, the same idea applied to raters rather than to anchors, has the strongest evidence base of any rater training method. You are running a cheaper version of it.

Step five: attach one data source per competency and kill the rest

A framework that no instrument feeds is a poster. For each competency, name exactly one primary source of evidence and one secondary. Then delete the forms that feed nothing.

The map matters because it finds the duplicates. Most L&D teams run three instruments that all measure manager behaviour: the performance form, the 360, and the engagement survey's "my manager" block. Pick one as primary and retire or repurpose the rest.

This is also the honest test of whether your framework is a selection tool or a development tool. If the primary evidence for a competency is a manager's observation, you can develop against it. You cannot use it to hire. Selection needs a scored instrument with a criterion study behind it, which is a different build.

A worked example, with illustrative numbers

A distribution company, 12 branch managers, an existing framework of 14 competencies with five levels.

They cut to nine competencies: four core, four for branch managers, one specialist. They rewrote 45 anchors into 36, dropping the level 5 and the level 1 entries that no one could describe. Retranslation with six managers survived 24 of the 36 anchors. Six anchors failed the competency sort, four sat more than one level apart, and two were filed identically under two competencies, which is how they discovered that "Customer focus" and "Commercial judgement" were the same behaviour in their business.

The nine competencies became seven. Two forms retired. The review cycle, previously six weeks of chasing, ran in two weeks because the rating conversation had something concrete to argue about.

None of the numbers here are benchmarks. They are what one company got. Yours will differ, and the direction of the difference is the useful part.

The honest limits

This method improves agreement between raters. It does not establish that your competencies predict performance. That requires a criterion study: measure the competency, measure an independent outcome like sales, error rate or supervisor rating, and look at the correlation. Bartram's eight factors hold up across 29 studies because someone did that work repeatedly. Your local framework holds up because your managers agree on it, which is a weaker and still useful claim.

It also does not fix a broken performance management system. If ratings are tied to pay rises in a fixed budget, managers will rate to the budget regardless of how good your anchors are. Anchors reduce noise. They do not overcome an incentive that rewards a particular distribution.

And four levels may be too many. Under about 150 employees, or where a role has one incumbent, two levels and a conversation is the honest design.

What to do this week

Open your framework and count the competencies. If the number is above twelve, cut, and do the cutting with the business in the room rather than in a review meeting afterwards.

Then book one afternoon with six managers, print the anchors without labels, and run the sort. Whatever survives is your framework. Whatever does not was never going to survive a review conversation either.

If you are rebuilding the whole assessment stack around a framework, the feature list is the honest place to start, and the comparison page shows which instruments feed which competency. For the rating conversation itself, the 360 question design guide covers how to phrase items so they produce evidence rather than impressions. And if you are still deciding whether the gap you found is a training need at all, the TNA versus TNI distinction is the cheaper read before you spend the budget.