The Due Process Stack: 3 Artifacts Every Public-Sector AI System Needs
Series Auditing the algorithm. Part 6. Due process in automated government decisions: notice, appeals, human review, and what public agencies can learn from Canada
Last week’s essay made the case against automated government without due process. Robodebt showed what happens when a public system can produce decisions at scale but cannot explain them, review them meaningfully, or reverse them in time.
This week is about the other side of the problem.
If due process is going to survive automated decision-making, it cannot remain a legal principle floating outside the system. It has to be built into the workflow. And that means three concrete artifacts matter more than most public-sector AI debates admit: a decision notice, an appeals pathway, and a human review protocol. Canada’s federal Directive on Automated Decision-Making is useful here because it treats explanation, recourse, human intervention, impact assessment, and ongoing monitoring as operational requirements for public administration, not abstract ethical aspirations.
1. The Decision Notice
Most public systems already notify people of outcomes. Very few actually explain them.
That difference matters. A notification says: your application was denied. A due-process notice says: this was the outcome, this is the basis, this is the role the system played, these are the main factors, and this is how you can contest it. Canada’s directive requires institutions to provide notice that a decision was made or assisted by an automated system, to explain in plain language how and why the decision was made, and to inform clients of relevant recourse options.
That is already more concrete than much of the global debate.
It means the notice is not just a communication layer added at the end. It is an operational artifact with mandatory fields. At minimum, a serious notice should include:
The outcome in plain language.
The legal or administrative basis.
Whether the system made, supported, or recommended the decision.
The main factors behind the result.
The relevant data categories used.
The date the decision takes effect.
The route and deadline to appeal.
A responsible office or contact point.
This is where many systems quietly fail. They can generate results at scale, but not explanations a person can act on. Once that happens, due process is already weakened at the first point of contact.
2. The Appeals Pathway
A complaint channel is not the same thing as an appeals system.
An actual appeals pathway has stages, responsible actors, escalation rules, time limits, and reversal authority. It shows where disagreement goes after the initial decision and what kind of review is possible at each stage. That is the only way recourse becomes real rather than decorative. Public-sector safeguards increasingly assume that when automated systems affect rights, benefits, or access, the affected person must have a meaningful and timely way to contest the outcome.
In practice, the pathway should answer four basic questions:
Who receives the first appeal?
What information can that reviewer access?
Who can pause, reverse, or escalate the case?
When does the case leave the original operational team?
Canada is useful again here not because it solves every ambiguity, but because it forces institutions to think about recourse as part of service design. That is a major shift. Too many public systems still treat appeals as something external to the workflow, as if a right on paper could compensate for a process that is opaque by design.
If no one can map the route from complaint to remedy, then the right to appeal exists mostly in theory.
3. The Human Review Protocol
This is the point where many systems claim safety and deliver theater.
Public agencies often say there is human oversight when what they really mean is that a person can approve the output of the system. But nominal human presence is not the same as meaningful review. The real question is whether that person has enough information, enough authority, and enough institutional protection to depart from the automated result when necessary. The logic behind Article 22 of the EU GDPR and related public-sector approaches is clear: what matters is not simply human involvement, but meaningful human intervention tied to the actual decision.
A real human review protocol should specify:
What the reviewer can see.
When review is mandatory rather than optional.
Whether enforcement can be paused while review occurs.
Whether the reviewer can override the result without penalty.
How overrides are logged and monitored.
When repeated anomalies trigger escalation.
This is where due process stops being a legal abstraction and becomes institutional design. A reviewer who cannot understand the basis of the outcome, cannot access the relevant trace, or is discouraged from reversing the result is not protecting anyone. They are legitimizing the system after the fact.
Why These Three Matter
These artifacts matter because they convert procedural rights into operational capacity.
The decision notice turns opacity into an explanation the person can respond to. The appeals pathway turns “contact us” into an actual route. The human review protocol turns symbolic oversight into authority. Together, they move due process from outside the system to inside it, where it has to live if automated public administration is going to remain contestable in practice.
That is also why this issue is bigger than procurement.
Some governments buy these systems. Others build them in-house. But the test is the same in both cases: if the system cannot explain itself, be reviewed, and be reversed, then it does not have due process designed in.
If your system can affect a person’s rights, benefits, or access, ask one hard question: can it explain itself, can it be reviewed, and can it be reversed?
Selected sources
European Parliament & Council of the European Union. (2016). Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data (General Data Protection Regulation). https://eur-lex.europa.eu/eli/reg/2016/679/oj
Government of Canada, Treasury Board of Canada Secretariat. (2024). Directive on Automated Decision-Making. https://www.tbs-sct.canada.ca/pol/doc-eng.aspx?id=32592
Office of the Australian Information Commissioner / Royal Commission into the Robodebt Scheme. (2023). Report of the Royal Commission into the Robodebt Scheme. https://robodebt.royalcommission.gov.au/publications/report
Royal Commission into the Robodebt Scheme. (2023). Overview of final report. https://www.apra.gov.au/sites/default/files/2024-02/Robodebt%20Royal%20Commision%20-%20Overview%20of%20Final%20Report.pdf
European Data Protection Board. (2018). Guidelines on automated individual decision-making and profiling for the purposes of Regulation 2016/679. https://www.edpb.europa.eu/our-work-tools/our-documents/guidelines/automated-decision-making-and-profiling_en
Information Commissioner’s Office. (2025). Rights related to automated decision making including profiling. https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/individual-rights/individual-rights/rights-related-to-automated-decision-making-including-profiling/
The human review protocol section names the real problem precisely: nominal human presence is not meaningful review. A reviewer who can see the output but not the reasoning trace isn't oversight — they're a liability shield.
What you've mapped is the institutional layer. There's a parallel problem one level down, at the model level, that your three artifacts assume has already been solved: the system has to be capable of explaining itself before any notice, pathway, or review protocol can function. If the model routes a high-consequence query to a cheap output path and never flags it, the human reviewer has nothing to work with. The audit trail is already broken before the institution touches it.
The model-level equivalent of your three artifacts: a routing architecture that treats irreversible-consequence domains as hard triggers (not soft signals), an output constraint that prohibits inferred-safety language absent grounded evidence, and a logged audit record that distinguishes forced escalations from standard processing. Without those, the due process stack you've described is sitting on a foundation that can silently fail.
Good series. The Robodebt anchor is the right one.
Marcela — the three-artifact framework turns due process from a legal principle into an engineering checklist, and that's exactly what's been missing from this conversation. Most public-sector AI debate stays at the level of "we need accountability" without specifying what accountability looks like as a workflow. You specified it.
The human review protocol section is the sharpest part. "Nominal human presence is not meaningful review" — that distinction is doing enormous work. A person who can see the output but not the reasoning, who can approve but not override without institutional friction, who exists in the loop on paper but not in practice — that person isn't oversight. They're a liability shield wearing a name badge.
I build custom AI agents for private-sector businesses and the same principles apply at a smaller scale. Every agent I build has scoped permissions, an escalation protocol, and a kill switch — because an agent that can act but can't be reviewed, paused, or reversed isn't a tool. It's a liability. The difference between public and private is the stakes, not the architecture. Your three artifacts could be a universal design standard, not just a government one.
The Robodebt anchor is the right one. Not hypothetical harm — documented harm. Decisions at scale with no explanation, no meaningful appeal, no human who could reverse them in time. That's what happens when the system is designed for throughput instead of contestability. Glad someone's writing the blueprint for the alternative.