‹ Culture 3.14 · What we ask ourselves
Question · Decisions
What data was actually available when the decision was taken?
Reconstructing a decision with today's database produces an unfair judgement and an optimistic model.
Why it matters
Tables are updated in place. A field corrected last week appears as though it had always been correct, and with it disappears the reason why somebody decided what they decided.
This affects two things at once: the evaluation of people, and the training of the models that learn from them.
What we know so far
Recording the state of the information at the moment of the decision, and not only its final value, is the only way we have found of answering this. In projects with event storage it comes almost for free; everywhere else it has to be decided before building.
When it is missing, the model’s temporal validation becomes an optimistic estimate and does not look like one.
What remains open
We do not know how much of this history is worth keeping, or for how long, once the volume grows.
Nor have we resolved how to reconstruct the availability of information in third-party systems that do not keep versions.