HRMIM
Is Your KPI Actually Measuring Performance? Construct Validity in Hotels

Is Your KPI Actually Measuring Performance? Construct Validity in Hotels

Beykan Çizel · Academic· 20 May 2026· 11 min read

The scorecard says "Guest Satisfaction." The value next to it is 87. The board reads that number and decides something — releases a bonus, puts a department on notice, defers a refurbishment.

So what is 87 a number of?

Open the formula and you usually find one of two things: the share of guests who completed the post-stay survey, or the average score of those who did. Neither is guest satisfaction. One is a participation measure; the other is the mean of a self-selected subset. Both can be useful. Neither is what the row label promises.

Psychometrics has a name for this problem: construct validity. Cronbach and Meehl (1955) formulated it for tests claiming to measure an attribute that is not operationally defined. Anxiety, job satisfaction, service quality — none of these come off a ruler; each is inferred from indicators. Occupancy is a countable quantity. "Guest satisfaction" and "team engagement" are constructs, and constructs are only ever measured indirectly.

On a countable quantity, error is measurement error. On a construct, error is about how well the indicator stands in for the concept. This piece deals with the second kind: how to see the distance between what an indicator is called and what it captures, how to close it, and what to do when you cannot.

Reporting survey completion rate as guest satisfaction

A resort sends an automated survey 24 hours after check-out. Monthly sends: 4,000. Responses: 460. The value reaching the scorecard is the mean of those responses on a five-point scale — 4.35 — reported on the "Guest Satisfaction" line.

Two separate problems are stacked in there. The first is coverage: roughly one guest in ten stands in for all of them. The second is selection: respondents are not a random sample. Delighted guests and furious guests answer. The large indifferent middle stays silent.

Put those together and the indicator moves for reasons that have nothing to do with the stay. Shift the send time and 4.35 moves. Ask the front desk to prompt guests at check-out and it moves again. Nothing about the guest experience has changed.

An indicator that moves for reasons other than the thing it claims to measure is showing you the clearest possible signature of a validity problem.

The same failure shows up on the employee side, where survey completion rate climbs while nothing on the floor changes.

What construct validity is, and what carries over from psychometrics

Cronbach and Meehl's 1955 paper rests on one question: to what extent does a score represent the theoretical construct it claims to measure? The answer is never a single coefficient. Validity is a judgement supported by accumulated evidence.

Messick (1995) reframed validity as a unified concept and pulled the consequences of score use into the argument alongside score interpretation. That second half lands hard in hospitality. An indicator is not merely reported — it is attached to bonuses, promotions and budget. How a number is used changes the number.

Kane (2013) proposed treating validity as an argument: every link in the chain from score to decision has to be defended on its own. On a hotel scorecard there are at least four inferential steps between "survey mean of 4.35" and "the front office performed well," and each one can break.

Three things carry over:

  • The construct is bigger than the indicator. An indicator samples the construct; it is not the construct.
  • Validity is a property of the interpretation, not of the number. The same figure can be valid in one context and invalid in another.
  • Evidence accumulates. No single check establishes that an indicator is valid.

What psychometric measurement does and does not capture gets asked routinely of HR instruments. Operational indicators deserve the same interrogation and rarely get it.

Content validity: does the indicator cover the whole concept

Content validity asks whether the indicator represents the full domain of the concept. Lawshe (1975) proposed an expert-panel ratio to systematise that judgement. The full method is heavier than a hotel scorecard needs; the logic transfers cleanly.

Write out the domain of "guest satisfaction": booking and check-in flow, room cleanliness and maintenance, food and beverage, staff attentiveness, facilities, noise and comfort, speed of problem resolution, perceived value for money.

Now mark how many your indicator touches. A single-question "overall satisfaction" mean touches all of them faintly and distinguishes none. An indicator built only on the housekeeping score measures a narrow slice under a name that oversells it.

Parasuraman, Zeithaml and Berry (1988) modelled service quality as a five-dimensional construct and argued for measuring the dimensions separately. If the concept is multi-dimensional, a one-dimensional indicator cannot represent it.

Practical rule: list the components, tick the ones the formula covers, and if coverage is under half, narrow the name. Not "Guest Satisfaction" — "Housekeeping Satisfaction." Renaming is the cheapest validity improvement available to you.

The proxy-metric trap: measuring what is easy to measure

A proxy is a measurable quantity standing in for the one you actually care about. Using proxies is not a flaw — it is unavoidable. The flaw is forgetting that the proxy is a proxy.

The common hospitality set:

What you want to measureThe proxy you useWhat the proxy misses
Guest satisfactionSurvey meanThe majority who did not respond
Service qualityComplaint countThe guest who never complains and never returns
Team engagementTurnover rateThe employee who stays because they have to
Training effectivenessTraining hoursThe part that reaches behaviour
Departmental productivityOutput per headThe quality traded away to get it

Ridgway (1956) argued that leaning on a single performance measure produces dysfunctional behaviour in the system being measured — and he did not stop there: the same paper shows that multiple and composite measures generate distortions of their own. Smith (1995) set out an eight-part typology of unintended consequences from publishing performance data in the public sector. The first is a focus on the measured dimension at the expense of the unmeasured one — tunnel vision, in his terms.

The moment a proxy is tied to reward, it stops being a proxy and becomes the target. If complaint count drives the bonus, not logging the complaint is also a way to hit the number.

Practical rule: next to every proxy, write the answer to "what does this miss?" — then check whether that answer can become a second indicator.

Two indicators with the same name and two different formulas

In multi-property groups the validity problem takes a second form: two properties use the same indicator name and two different formulas. "Employee Turnover" appears on both scorecards.

  • City hotel: (permanent staff leaving in the period / average permanent headcount) × 100
  • Seasonal resort: (all staff leaving in the period / opening headcount) × 100

The second formula sweeps in planned end-of-season departures. These two numbers cannot be summed, averaged or compared. Because they share a label, they get summed, averaged and compared anyway.

The measurement literature does treat it as an assumption to be tested rather than assumed that an instrument measures the same construct the same way across different groups; it calls that measurement invariance. What these two properties have is cruder than that. Invariance testing already presumes both groups are using the same instrument. Two formulas are not one instrument, so there is no invariance left to test — the shared definition has to come first.

This is a definition failure, not a data failure, and no amount of cleaning touches it. The only fix is making the formula a required field alongside the name. As long as the label is free text and the formula lives in someone's head, two properties will keep using one word to mean two things.

Measuring the distance between an indicator's name and its formula

So far the distance has been an abstraction. Make it visible with a four-line card for every indicator:

  1. Name — the label that appears on the scorecard.
  2. Construct — the concept this indicator claims to represent, in one sentence.
  3. Formula — numerator, denominator, unit, period.
  4. Out of scope — the part of the concept this formula does not reach, stated plainly.

Line four does the work. Left blank, the card is decoration. Filled in honestly, the tension between "Name" and "Out of scope" becomes visible to everyone who reads it — including whoever defends the indicator.

On the evidence side, Campbell and Fiske (1959) framed convergent and discriminant validity through the multitrait-multimethod matrix: indicators claiming to measure the same construct should move together, and indicators measuring different constructs should not.

Applying that in a hotel costs almost nothing. Does your guest satisfaction indicator move with repeat-stay rate? Is it consistent with your review-platform scores? Consistency is convergent evidence. Inconsistency demands an explanation — and "data lag" is usually not the real one.

One indicator or a set of them

The pull toward a single number is strong. The board wants one figure, the deck carries one, the scorecard shows one row.

Reichheld (2003) proposed a single-question recommendation measure as an indicator of growth. Keiningham and colleagues (2007) retested that claim against Norwegian customer satisfaction barometer data and could not reproduce the measure's "clear superiority" over the alternatives. Who wins that argument is secondary. The point is that the representational claim loaded onto a single indicator is an empirical claim, and empirical claims can be tested.

Ridgway's (1956) warning applies again, and it cuts both ways. Single measures narrow behaviour; in the same paper Ridgway shows that multiple and composite measures create problems of their own. Moving to an indicator set does not dissolve that problem. It relocates it. Kaplan and Norton's (1992) balanced scorecard was built as a multi-dimensional answer to the narrowing that financial measures produce on their own.

The workable middle ground: one indicator set per construct, one composite row on the scorecard. Three or four indicators in the set, the composite as their weighted total, and — the part that gets dropped — the components always expandable.

If the composite falls and you cannot see which component fell, what you have is not an indicator. It is a lid.

Reliability and validity keep getting confused

These are not the same thing, and confusing them is expensive.

Reliability is consistency: measure again under the same conditions and do you get the same result? Cronbach (1951) defined the alpha coefficient to estimate the internal consistency of a scale.

Validity is accuracy: are you measuring the right thing?

A bathroom scale makes the distinction obvious. A scale that reads three kilos heavy every single time is perfectly reliable. It is also wrong.

The hospitality version: if your survey completion rate lands in the 11-12% band every month, that indicator is highly reliable. It still does not measure guest satisfaction. A stable number in a narrow band is not validity evidence — usually it is only evidence that the process is stable.

The relationship runs one way. Reliability is a precondition for validity, not a guarantee. An unreliable indicator cannot be valid; a reliable one can be thoroughly invalid.

For a worked case, the validity comparison of two instruments runs this same distinction through personality inventories.

Five questions before an indicator earns a place on the scorecard

Staple this to the indicator card. Nothing enters the indicator layer of the goal tree until all five have answers.

  1. What construct does this represent, and can I define it in one sentence? If you cannot, the indicator represents a habit rather than a concept.
  2. Which part of the construct does the formula cover, and which part does it leave out? If the excluded part is not written down, the question is unanswered.
  3. Can this indicator move while the construct stays still? If changing the survey send time moves the number, the indicator has an exposed surface someone will eventually use.
  4. Does it move consistently with another indicator measuring the same construct? Without convergent evidence you need, at minimum, an explanation.
  5. What decision attaches to it? If none does, it belongs on a monitoring dashboard, not the scorecard.

Question five eliminates the most indicators, and usually the right ones. A number can be interesting. To earn a scorecard row it has to change a decision.

What it costs to remove an invalid indicator

Detecting an invalid indicator is easy. Removing it is not, and the cost shows up in four places.

The trend breaks. A three-year series ends. That is a genuine loss, not something to wave away.

Comparability breaks. This period cannot sit beside the last one. Either you restate history under the new definition or you mark the break explicitly. There is no third option.

The owner pushes back. Every indicator has an owner, and that owner's bonus is frequently attached to the row. A conversation about validity reads to them as a conversation about their performance.

Weighting reopens. Remove an indicator and the total no longer reaches 100. Weights get redistributed, and a settled argument restarts.

None of this justifies keeping an invalid indicator. It justifies planning the removal. The lowest-friction route is to run the old and new indicators in parallel for one period, measure the gap, and explain the change with that gap. An indicator deleted quietly does not end the argument — it postpones it a quarter.

Making the validity check recurring, not one-off

Validity is not a permanent property of an indicator. When the context shifts, the evidence goes stale.

An indicator valid under the conditions of the year it was built can be invalidated by any of these: the channel mix moves, a property joins the group, the PMS changes, the survey tool changes, the guest profile changes, or the indicator gets attached to a bonus. The last acts fastest.

Kane's (2013) argument-based approach is useful for a practical reason: validity is not a certificate you earn once and file, it is a case you keep making.

A cycle that actually gets done:

  • Annually, review every indicator card — are the construct, formula and out-of-scope lines still accurate?
  • Quarterly, check convergent evidence — are the related indicators still moving together?
  • On every structural change — new property, new system, new survey tool — pull the affected indicators for an off-cycle review.

The precondition for any of it is that indicator definitions sit in one accessible place. If each property keeps its own file, no review is possible — there is no reference to compare against. Keeping indicator definitions in one place turns the validity check from an annual project into routine maintenance.

Conclusion

An indicator's name is a claim. The label "Guest Satisfaction" asserts that the number underneath represents guest satisfaction. Sixty years of validity literature reduces to one sentence: that claim requires evidence.

The evidence is not expensive. Write the construct in one sentence, record the formula, state what falls outside it, check consistency against related indicators. That is not a research project — it is record-keeping discipline.

Hold three things apart. Reliability is consistency. Validity is accuracy. Usefulness is independent of both. An indicator can be reliable, useful and invalid at once, and those are the most dangerous rows on any scorecard: stable, cheap to calculate, familiar to everyone, and measuring the wrong thing.

Before your next scorecard review, take one indicator and write its name, construct, formula and out-of-scope line side by side. The distance is usually wider than you expected — which is not bad news. It is a problem that has finally become measurable.


If an outside read on your indicator definitions, formulas and owners would help, the HRMIM team works with hotel groups on measurement structures regularly. To talk through a single row on your own scorecard, get in touch.

Get New Posts by Email

We will email you when a new blog post is published.

Loading the form…

Related Posts

Let's work with HRMIM

Explore our platform free of charge.