What a 360 cannot see: work the rater never watches

A 360 only scores work its raters actually witness. Map each competency to who can observe it, and route the unobserved ones to a different evidence source.

Rajendra Kumar · 20 November 2026 · 9 min read

A wide view from a mezzanine over a sunlit Indian factory floor of machines and workbenches, with one small figure seated inside a glass-walled site office at the far end

Representational image.

The report said 4.4 out of 5 on strategic thinking, from six peers and four direct reports. The plant head read it and said nothing, because he knew where the number came from. The operations manager it described ran the weekly production review in front of all ten raters. He was crisp in that room. He asked good questions. He looked strategic.

The decisions that actually mattered that quarter happened somewhere else. Which line to shut for a week. Whether to hold a marginal supplier or drop them. The headcount trade-off between maintenance and quality. Those happened in a closed room with the finance controller and the plant head, or alone at 2am when a line went down. Not one of the ten raters was in any of those rooms. They rated the meeting they could see, and the form averaged it into a score for work they never watched.

This is not a wording problem or a rater-bias problem. It is a coverage problem, and almost no 360 programme checks for it. The instrument scores what its raters witness. For a manager, that is a small and non-random slice of the job.

A 360 is a witness survey, not a performance record

Start from what the instrument actually is. You ask a set of people to report behaviour they have seen. A rater who has watched someone coach a junior for six months has real data. A rater who has never sat in a budget call has no data on commercial judgement, whatever the form asks.

Two findings make this concrete.

The first is how little different sources agree. In Conway and Huffcutt's meta-analysis of multisource ratings, the uncorrected correlations between rater groups ran from .14 for self against subordinate up to .34 for peer against supervisor. A correlation of .14 means two sources share about two percent of their variance. Part of that gap is bias. But part of it is simpler: different sources literally watch different work. Scullen, Mount and Goff found that a meaningful share of the difference between boss and subordinate ratings is perspective-related, meaning it tracks real differences in what each group is positioned to see, not just idiosyncratic error.

The second is that opportunity to observe is a precondition, not a nicety. Nowack and Mashihi, reviewing the evidence for practitioners, list different opportunities to observe as one of the core reasons raters disagree, and they note that the right rater set depends on the competencies being assessed and on who actually had the chance to see them. The standard assumption behind pooling ratings is that every rater had an opportunity to observe the performance being rated. When that assumption fails for a whole competency, the mean is not a measurement. It is an average of guesses.

Here is the mechanism that does the damage. A rater who has not observed a behaviour does not leave the item blank. The form demands a number, so they supply one, built from the nearest thing they have: a general impression, a reputation, the halo of the meetings they did attend. The result is a confident 4.4 on strategic thinking, produced by people who have never seen a strategic decision. The number looks like data. It is inference wearing data's clothes.

And a 360 is a weak proxy for observed work even at its best. Atkins and Wood compared 360 ratings against an assessment centre, where trained assessors watch people do real exercises. Across 63 team leaders, the overall assessment-centre score correlated only .39 with the averaged supervisor, peer and subordinate rating. Most individual rater-to-exercise correlations were not significant. If a pooled 360 only tracks observed work sample performance at .39, an unobserved competency rated by the wrong people is tracking almost nothing.

Build the observability map before you write a single item

The fix is not better wording. It is deciding, competency by competency, whether the rater pool can see the work at all. Do this before the form is built, in the order the work actually happens.

Step 1. List the competencies the cycle will score. Take the framework as it stands.

Step 2. For each competency, write the real events where it shows up in this role. Not a definition. The actual meetings, calls and moments. For decisiveness in an operations manager: the weekly production review, the capex approval call with finance, the 2am breakdown decision. Three events, three different audiences.

Step 3. Name who is in the room for each event, or who holds the record. This is the set of legitimate witnesses for that competency. A peer who attends the production review is a witness to decisiveness there. The same peer is not a witness to the capex call. The finance controller is.

Step 4. Mark each competency by its coverage. Witnessed: enough raters across the pool see the events, so a score is defensible. Partly witnessed: only one source sees it, so only that source should rate it. Unwitnessed: nobody in the rater pool sees the events that matter.

Step 5. Route the unwitnessed and partly witnessed competencies out of the scored 360. They do not become a number averaged from people who were not there. They go to a different evidence source: a work sample, a review of the actual decision log, the boss's structured account of specific decisions, or an assessment exercise. The 360 keeps the competencies it can legitimately measure.

CompetencyWhere it shows upWho sees itVerdictEvidence source
CoachingWeekly 1:1s, floor feedbackDirect reportsWitnessed360, upward group
CollaborationCross-shift handover, project reviewPeers, reportsWitnessed360, peer + upward
CommunicationProduction review, town hallAll groupsWitnessed360, all groups
DecisivenessProduction review; capex call; 2am breakdownReview: all. Capex: boss + finance. Breakdown: shift leadPartly witnessed360 for review; decision-log review for the rest
Commercial judgementSupplier negotiation, budget trade-offBoss, finance controllerUnwitnessed by poolDecision-log review with boss
Strategic prioritisationQuarterly plan, line-shutdown callBoss, plant headUnwitnessed by poolWork sample: a real planning case
Crisis judgementNight breakdown, safety incidentShift lead, safety officerPartly witnessedIncident-record review
Integrity under pressureSupplier gift offer, reporting a near-missAlmost nobodyUnwitnessedStructured boss interview, records

Read the last column, not the verdict. The point of the map is that every competency ends up with an evidence source that matches who can actually see it. The 360 is one source among several, not the whole assessment.

A worked example, with illustrative numbers

Numbers here are illustrative. The structure is not.

An operations manager at a components plant goes through a cycle scoring the eight competencies above. The original form sends all eight to six peers and four direct reports. Strategic prioritisation comes back at 4.4, commercial judgement at 4.1. The plant head knows both numbers describe the production review, because that is the only strategic work the raters have ever seen.

Run the map. Five competencies are witnessed by the pool and stay in the 360. Decisiveness and crisis judgement are partly witnessed, so the 360 scores only the slice the pool sees, and the rest goes to a decision-log and incident-record review. Commercial judgement, strategic prioritisation and integrity under pressure are unwitnessed by peers and reports, so they leave the scored form entirely and go to a work sample and a structured boss review.

CompetencyBefore: score from poolAfter: where it is scored
Strategic prioritisation4.4 from 10 ratersWork sample, assessed by plant head
Commercial judgement4.1 from 10 ratersDecision-log review with boss
Integrity under pressure4.6 from 10 ratersStructured interview + records
Coaching3.2 from 4 reports360, unchanged
Communication4.0 from 10 raters360, unchanged

The report now has five scored 360 rows instead of eight, and three competencies assessed by people who actually saw the work. The witnessed competencies were already measured well, so little else changes. What changes is that the report no longer prints a confident number about work nobody watched. Strategic prioritisation now reads as an assessed planning case, not an echo of a meeting.

This is the same discipline as deciding who should not be on a rater list, applied one level up. Exclusions remove raters who should not rate the person. The observability map removes items nobody can rate at all. Both protect the arithmetic by refusing to average things that are not observations.

The honest limit

The map is a judgement, and you can get it wrong. Assigning observability is not itself a measurement. Test it the cheap way: ask two raters to describe the last specific time they saw the person do the behaviour. If they cannot name an event, the item is unwitnessed for them, whatever your map says. That question is the real gate.

The field disputes how much of the gap is perspective. Viswesvaran and Ones, cumulating the peer-supervisor literature, found construct-level convergence for most performance dimensions, meaning peers and supervisors are largely rating the same thing and the disagreement is mostly measurement error rather than different work seen. So do not overclaim that every cross-source gap is "different work witnessed". The map does not depend on winning that argument. Opportunity to observe is a precondition for a rating to mean anything, whichever side of the debate you sit on.

Dropping items can look like hiding weakness. A sponsor who bought a full leadership dashboard will ask why three competencies disappeared from the 360. The answer is that they did not disappear, they moved to a source that can actually see them. Say that out loud and show the replacement evidence, or it reads as a cover-up.

Sometimes the fix is to widen observation, not drop the item. If commercial judgement matters and only the boss sees it, invite a peer into the supplier call or open the decision log. Making the work witnessed can be cheaper and better than routing it away. The map tells you where the hole is; it does not force you to fill it by subtraction.

And the limit that sits under all of it: sizing the rater pool properly still comes first. A witnessed competency rated by two people is still a coin toss. The map fixes coverage; it does not fix a group that is too small to mean anything.

Two columns to add this week

Take the competency list for your next cycle and add two columns: last observed event, and who saw it. Fill them from the role's actual calendar and records, not from the framework's definitions. Any row where you cannot name a specific event and a witness comes out of the scored form and into a different evidence source, a work sample, a decision-log review, a structured interview.

That is an afternoon of work, and it is the difference between a report that describes a manager and a report that describes the meetings the manager's raters happened to attend. If the form cannot carry a "did not observe" option and route items by who can see them, that is a constraint on the tool, and it is worth fixing before the next window opens. A 360 that admits what it cannot see is more useful than one that scores everything confidently. The blind spot is not only in the manager. It is in the instrument, and you can map it.

Next step

Run the same measurement on your own team

Bring a real team and a real competency. We set up the assessment, run one cycle, and walk you through the report so you can judge the output yourself.

Written by

Rajendra Kumar

Head of Enterprise Business, Conceptial

Runs enterprise work at Conceptial, the training and consulting firm behind BolderBrain. Two decades of building assessment and development programmes for Indian enterprises.