Missing Data or Not-Yet-Due? Measurement Discipline in Peak Season
Mid-June. The property is full, the roster is complete, the front desk starts at six. The general manager opens the May scorecard and finds a zero next to one indicator: employee engagement index. That indicator is measured once a year, in November. The zero in May is not a performance result. It is an artefact the measurement system produced on its own.
KPI measurement frequency is as consequential a design decision as weighting, and it almost never gets the same attention. How often an indicator is measured, how it appears on the scorecard in the months it is not measured, and how that blank feeds into the score — a scorecard built without answering those three questions will throw false alarms in the middle of the season. This piece takes them in order.
What a manager thinks when the July scorecard shows a zero
A zero is not a neutral mark to the person reading the scorecard. Numerically, "no value" and "value of zero" are plainly different. Visually they sit in the same cell and look identical. The manager does not see a measurement gap. The manager sees a failure.
What follows is predictable. The department head gets a call, explains that the indicator is not measured in July, and the matter closes. Next month the same conversation runs again. Ridgway made the general point decades ago: the measurement system itself produces behaviour — Ridgway, V.F. (1956), Dysfunctional Consequences of Performance Measurements, Administrative Science Quarterly 1(2), 240-247.
The cost is not the explaining. It is trust. A manager summoned to defend the scorecard three times for the wrong reason stops taking it seriously on the fourth.
The second cost is quieter. The property score drops on a decline that never happened, and that score travels: into group consolidation, into the property manager's review, sometimes into a bonus calculation. The fewer people who remember where the zero came from, the more the number becomes the truth.
Missing data and not-yet-due data are not the same thing
Both show up as a blank cell. Conceptually they are nowhere near each other.
Missing data is a value that should have been measured and was not. The measurement window opened, nobody entered a number, or the source system did not run that day. There is a real gap, and the gap has a cause.
Not-yet-due data is not a gap at all. A quarterly indicator with no June value is evidence that the measurement calendar is working. Nothing is absent; that cell was never designed to be filled.
The academic spine for this distinction is Rubin's missing-data typology — Rubin, D.B. (1976), Inference and Missing Data, Biometrika 63(3), 581-592. Rubin shows that why a value is missing determines how the analysis must treat it: missing completely at random (MCAR), missing at random given observed variables (MAR), and missing not at random (MNAR). The scorecard translation is blunt. If you fill a blank without knowing the mechanism behind it, the number you produce says more about your fill method than about the thing you meant to measure.
Per-indicator measurement frequency: monthly, quarterly, half-yearly, annual
Frequency is a property of the indicator, not of the scorecard. Four different frequencies can run on the same scorecard at the same time, and they should.
| Frequency | Why this frequency | Hospitality example |
|---|---|---|
| Monthly | Operational, flows automatically from a source system, can change a decision within the month | Occupancy, employee turnover rate, overtime hours |
| Quarterly | Meaningful movement takes more than one month to appear | Departmental training completion, internal service ratings |
| Half-yearly | Measurement is expensive; measuring more often just adds noise | Competency assessment, 360-degree feedback |
| Annual | Structural, and the measurement is itself an organisational exercise | Employee engagement index, employer brand study |
The performance measure record sheet developed by Neely and colleagues carries frequency as one of the fields to be completed for every measure — Neely, A., Richards, H., Mills, J., Platts, K. & Bourne, M. (1997), Designing performance measures: a structured approach, International Journal of Operations & Production Management 17(11), 1131-1152. The literature review the sheet was built on is the earlier paper — Neely, A., Gregory, M. & Platts, K. (1995), Performance measurement system design: a literature review and research agenda, International Journal of Operations & Production Management 15(4), 80-116. Frequency belongs to the definition, alongside the name and the formula.
Which indicators hotel general managers actually use, and how they interpret them when deciding, is a separate line of research — Harris, P.J. & Mongiello, M. (2001), Key performance indicators in European hotel properties: general managers' choices and company profiles, International Journal of Contemporary Hospitality Management 13(3), 120-128. That study addresses the contents of the indicator set, not how often it is measured. Frequency is a layer that has to be written on top of the set.
Who sets the frequency, and on what basis
Not the department that owns the indicator. The owner is accountable for taking the measurement; the scorecard design decides how often. Otherwise every department proposes a calendar shaped by its own workload, and the scorecard stops being comparable across properties.
Three criteria drive the decision.
How fast does the underlying thing actually move? Measuring monthly something that shifts meaningfully twice a year makes measurement error larger than signal.
What does the measurement cost? A number that flows automatically from a source system costs almost nothing monthly. A competency assessment across a hundred people costs operational time, and in peak season that time does not exist.
Does the measurement feed a decision? If nobody will act on the result, measuring it monthly does one thing only: it adds data-entry load.
All three should be answered alongside the question of what an indicator actually captures. Measuring a questionable indicator more often does not make it more accurate — it reproduces the same error more frequently.
Why writing zero is a measurement error
Typing zero into a blank cell looks like a data-entry convenience. Statistically it is one of the worst imputations available.
Zero-filling breaks two things at once. It pulls the period average down with a value that occurs nowhere in the real distribution. And it inflates variance artificially, so that when the real value arrives next period the system reads it as a dramatic improvement. Schafer and Graham set out how naive imputation produces both bias and false precision — Schafer, J.L. & Graham, J.W. (2002), Missing data: Our view of the state of the art, Psychological Methods 7(2), 147-177.
Measurement science frames the same split as trueness and precision: ISO 5725-1:2023 separates the accuracy of a measurement method into those two components. Zero-filling damages trueness, and the direction of the damage depends on the indicator's target direction. For a lower-is-better indicator such as turnover, a zero manufactures fake excellence.
The same objection applies to mean imputation, only more politely. Filling a blank with the last three months' average makes the indicator look inert and shrinks period-over-period difference artificially. Graham's review of what works under real operating conditions stresses that single-value imputation ignores uncertainty and therefore makes results look more definite than they are — Graham, J.W. (2009), Missing data analysis: Making it work in the real world, Annual Review of Psychology 60, 549-576.
Zero is a value. A blank is a state. Any system that writes both into the same cell has lost the thing it was measuring.
Three kinds of missing value
Not every blank cell is blank for the same reason. Separating the three matters for scoring and for follow-up.
Late
The window opened, the value has not been entered yet, but it will be. The cause is usually operational: the owner is on leave, the source report closes on the fifth, the accounting close slipped. Late data is a temporary state with a deadline attached.
Correct handling: mark the cell as pending, hold the indicator out of the period score, and recalculate the period when the value lands. Log the recalculation — otherwise two different scores for the same period start circulating.
Lost
The window closed and the value will never be entered. The survey did not run, the count was skipped, the source system produced nothing that month. This is a genuine gap and it cannot be recovered.
Correct handling: mark the cell as lost and record why. Lost values are the real subject of data-quality control; if they cluster, the problem is the process, not the indicator. Enders' applied treatment of missing data makes the point that the mechanisms function as assumptions for the analysis, that the assumption decides which method gives a valid answer, and that the mechanism itself cannot be verified from the data — Enders, C.K. (2010), Applied Missing Data Analysis, Guilford Press. The practical consequence: where the reason for a blank is known, it has to be recorded at the time, because it cannot be read back out of the data later.
Not yet due
This is not a gap. The indicator is not measured in this period; the cell is blank by design. It falls into none of Rubin's categories, because there is no unobserved observation — there is simply no observation defined for that cell.
Correct handling: mark it not-yet-due, remove the indicator from both the numerator and the weight denominator, and surface the next measurement date. What governs the treatment, in the frame Little and Rubin set out, is the mechanism — Little, R.J.A. & Rubin, D.B. (2019), Statistical Analysis with Missing Data, 3rd edition, Wiley. For a not-yet-due cell the mechanism is fully known and sits in the hands of the scorecard designer; there is no value to estimate, so no imputation is called for and the cell simply leaves the denominator.
How a missing value should be flagged in scoring
A few questions come up every time, and the answers are not ambiguous.
Does a not-yet-due indicator count towards the scorecard total? No. For that period it comes out of both the score and the weight denominator, and the remaining weights are renormalised to one hundred. Otherwise an unmeasured indicator dilutes the ones that were measured.
Can a lost value be scored as zero? No. A lost value is not scored; it is flagged and it lands in the data-quality report. What deserves scoring there is the process, not the indicator.
Does a late value trigger a recalculation? Yes, inside a defined window. A value arriving after the window closes stays lost and is reported in the following period. The window length belongs in the scorecard definition.
Does one missing indicator invalidate the whole period? No — but a threshold is needed. A period that falls below a set share of the indicators due should produce a data-quality warning rather than a score.
All of these rules should be written alongside the choice of scoring method. A renormalised weight denominator behaves differently for a closeness-to-target indicator than for a ratio-based one.
Holding collection discipline when the property is full
Between June and September nobody chases cells. This is not a discipline problem, it is a capacity problem. Expecting a front-desk supervisor to key numbers into a sheet on the third of the month at ninety per cent occupancy means the system does not understand the operation it sits on.
The link between high job demands and exhaustion is well established: the paper that set out the job demands-resources model found job demands to be related primarily to the exhaustion component of burnout, and a lack of job resources to the disengagement component — Demerouti, E., Bakker, A.B., Nachreiner, F. & Schaufeli, W.B. (2001), The job demands-resources model of burnout, Journal of Applied Psychology 86(3), 499-512. That study ran in human services, industry and transport samples; it carries no finding about a hotel season. Data entry falling away in August is not a result drawn from that literature. It is a plain observation about operational load on the floor: the measurement task cannot compete with the guest standing in front of you.
The fix is not more reminders. It is collecting less during the season. Three practical rules.
Keep only automatically sourced indicators on a monthly cadence through the season. Move anything requiring manual entry to a quarterly or off-season calendar. And for whatever must stay manual, push the deadline to the tenth of the month rather than the third.
With seasonal teams there is a training dimension too. If the person entering the data changes at the start of the season, the scheduling design for seasonal teams has to be planned at the same time as the measurement calendar.
What a data-control report has to cover
A second report should sit next to the score: the one that measures the data itself. The score answers "how are we doing". The control report answers "can we trust that number".
It has to carry the count of indicators due in the period, entered, late, lost, and not-yet-due — that last one on its own line, because folding it in with lost values corrupts the very thing the report exists to measure. Add the last and next measurement dates per indicator.
For formal outlier rules, the NIST/SEMATECH e-Handbook of Statistical Methods is a reasonable standard reference; its outlier section sets out the Grubbs, Tietjen-Moore and generalized ESD tests. The purpose of the control report is not to find someone to blame. It is to make visible which indicators are failing as a process.
How often it is read is part of its design. A monthly control report goes unread all season; a fortnightly summary showing only the rows that changed gets read.
Comparing with prior periods after the frequency changes
Moving an indicator from quarterly to monthly looks like an innocent improvement. For comparison purposes it is not.
The occasion of measurement is itself a source of error. Cronbach and colleagues' generalizability theory models measurement conditions — rater, item, occasion — as distinct error facets, and shows that changing one of them changes how far a score generalises — Cronbach, L.J., Gleser, G.C., Nanda, H. & Rajaratnam, N. (1972), The Dependability of Behavioral Measurements: Theory of Generalizability for Scores and Profiles, Wiley. On the internal-consistency side the classic reference is Cronbach, L.J. (1951), Coefficient alpha and the internal structure of tests, Psychometrika 16(3), 297-334.
The practical consequence: once frequency changes, prior-period data is not directly comparable. Either the change is marked as a break point and comparison restarts there, or historical data is re-aggregated to the new frequency — which only works if sub-period data already exists. There is no third option. Carrying on without documenting the change turns the trend into an artefact of measurement design.
The measurement calendar to set before the season starts
The measurement calendar is not built during the season. It is built in April and left alone until October.
The fields it needs are plain: indicator name, measurement frequency, measurement months, data source, owner, entry deadline, and the late window. With those seven filled in, "is this indicator due this month" stops being a matter of opinion.
Atkinson and Brander Brown's study of UK hotels finds that properties monitor performance in great detail and yet still lean heavily on traditional, short-term, backward-looking measures — Atkinson, H. & Brander Brown, J. (2001), Rethinking performance measures: assessing progress in UK hotels, International Journal of Contemporary Hospitality Management 13(3), 128-136. Detailed monitoring is not the same as a written design: an undocumented measurement calendar is another face of the same gap.
The calendar also has a technical precondition: a structure that can hold measurement frequency per indicator. As long as frequency lives as a single scorecard-level setting, none of the distinctions above can be applied.
Conclusion
A blank cell carries information, and the information changes with the reason it is blank. A not-yet-due indicator says the system is working. A lost one says the process is not. A late one says only that the calendar is tight. A scorecard that renders all three as the same zero does not merely erase that information — it substitutes a false signal for it.
Make measurement frequency a mandatory field in the indicator definition. Flag missing values by type. Take unmeasured indicators out of the weight denominator. Those three are the minimum discipline that keeps a scorecard credible in the middle of the season. Everything else — imputation methods, sensitivity checks, break-point logs — is built on top of them.
Preparation for next season should start with the measurement calendar rather than the scorecard design. In most groups what needs fixing is not the indicators, but the unwritten assumptions about when they get measured.
If you want the measurement calendar settled before the season starts, and you currently cannot see from one place which indicator is measured at what frequency across your properties, it is worth a conversation. The HRMIM team works on measurement and reporting discipline in hotel groups — write to us through the contact page and we will look at your current scorecard and calendar together.


