Justice Is the System Around Your Data: The Third Value of the DoGoDa Oath

Three criteria decided which students at DUO, the Dutch student grant agency, got flagged for a home visit: their type of education, the distance between their address and their parents’, and their age. Every student was measured by exactly the same rule. No machine learning anywhere in the process, just a set of criteria applied identically to everyone. In November 2024 the Autoriteit Persoonsgegevens ruled the whole approach unlawful. That verdict took a regulator and years to arrive. Nobody near the data had flagged it before then.

This is value 3 of 8 in the proposed data professional’s oath I have been reading and writing through, post by post: justice, right after dignity and interpersonal trust. At first sight justice looks a lot like fairness. If you ask me, the split between them decides what our job actually is.

The split I draw

Fairness lives within the data, the logic and the decisions they drive. Justice lives in the system around that. Treating everyone fairly by the same rule, and then using that system to keep people from a payment or accreditation that is rightfully theirs, is still injustice. Even inside the fairness layer, the reasonable definitions of fair provably contradict each other: Kleinberg, Mullainathan and Raghavan proved in 2017 that outside narrow, constrained cases, no method can satisfy several reasonable fairness conditions at once. So a team that treats the fairness layer as the whole assignment has stopped one layer short, and made an unexamined choice on the way there.

No model required

DUO’s criteria prove the point without a single model in the room. Type of education, address distance, age: three variables, applied the same way to everyone, no protected attribute anywhere in the formula. Students with a non-European migration background scored higher on exactly those three criteria and were checked and visited more often as a result. The Autoriteit Persoonsgegevens found no objective justification for the pattern and ruled the processing unlawful. Every layer of that system passed its own test. The rule was applied identically to everyone, and the whole was still aimed at keeping a grant away from students who were entitled to it.

The lookout, performed

Justice is also the layer where the story can turn around, and it did. In 2019 Science published Obermeyer and colleagues’ analysis of a US health algorithm used to route extra care. The algorithm scored every patient by one rule: predicted cost stood in for predicted need. At an equal risk score, Black patients turned out to be considerably sicker, because historically less money gets spent on their care at the same level of illness. Researchers outside the building noticed the gap, said so publicly, and worked with the developers on a better target variable. The bias dropped by 84 percent. A US health system in 2019 is not a Dutch BI team in 2026, and the setting does not transfer directly, but the mechanism does: fair scoring inside the model, injustice in what the system did with the score, and a lookout that worked because someone outside the room acted on a whiff.

The duty of our seat

Data literacy keeps spreading, and tools like TimeXtender and Microsoft Fabric put analytics engineering in the hands of a much wider group of builders than five years ago. The odds of creating something genuinely good with data are rising. So are the odds of feeding a system that does injustice. I do not think most of us set out to build the second kind, on the contrary. But history has proven us capable of building unjust systems anyway, and just as capable of catching and correcting them, the way the Obermeyer researchers did.

Here is where the duty lands: we are almost never in the room where these systems get invented. The scope, the target variable, the criteria a case worker acts on, those decisions usually sit one or two departments away from the data professional’s desk. So if we are going to swear an oath that includes justice, let it be this one: always be on the lookout for the systems our data products might be feeding, and act the moment we get a whiff that something is wrong.

Dignity, the first value in this series, said a person is never just a row in your data. Interpersonal trust, the second, said the numbers behind a decision must stay explainable to the person acting on them. Justice adds the layer around both: watch what your product feeds into, not just what your product does on its own terms. The DUO criteria passed their own test. The lookout was the test nobody ran.

Happy to think this through with you.