Artificial Intelligence Should Not Only Be Useful, but Legally Defensible
Generative AI, Privacy, and Bias: Risks We Can No Longer Ignore
Public debate about artificial intelligence often focuses on speed, creativity, and efficiency. But those are not the decisive tests. The more important question is whether an AI system can justify the effects it has on people, especially when those effects shape rights, duties, access, or exclusion.
That question becomes urgent in two overlapping fields. Amnesty International argues that many generative AI systems are built on data practices that invade privacy, reproduce discrimination, and threaten freedom of expression and thought. In parallel, research on tax administration shows that automated systems can become opaque instruments of public power when they influence audits, sanctions, eligibility, or procedural burdens without clear safeguards.
When Efficiency Becomes Power
AI is often presented as an efficiency tool. Yet in public administration, efficiency can quickly become a mechanism of power, because automated systems do not merely assist; they classify, rank, prioritize, and sometimes determine who gets attention from the state.
In tax administration, this matters even more because the relationship is not voluntary. Taxpayers must interact with the system, often without meaningful visibility into the criteria used to assess risk or trigger review. That combination of obligation, asymmetry, and coercive potential makes algorithmic opacity especially serious.
This is why the debate cannot stop at whether a model improves throughput or reduces administrative cost. A system may be technically effective and still produce outcomes that are ethically indefensible or legally fragile when challenged under principles of transparency, due process, proportionality, and non-discrimination.
The Human Rights Cost of AI Design
Amnesty International’s analysis makes a broader point that reaches beyond any single product. Many prominent generative AI systems, it argues, are built on mass web scraping and large-scale data extraction practices that collect personal and creative data without meaningful knowledge or consent from the people concerned.
According to the report, these design choices are linked to several downstream harms. They include privacy violations, systematic bias against marginalized groups, cultural and linguistic skew toward English-dominant contexts, heightened risks of overbroad censorship, and forms of automation bias that can distort user judgment.
That framing is important because it shifts the conversation away from isolated failures and toward architecture. The core issue is not only what an AI system produces at the end of the pipeline, but what assumptions, incentives, and data practices were embedded into the system from the start.
Why Tax Administration Is Different
Tax administration is a particularly sensitive case for AI governance because its decisions can have direct legal and financial effects. The MEAAT framework treats this domain as one of high-impact decision-making, especially where systems influence audit selection, automatic debt determinations, sanctions, suspensions of fiscal credentials, or denial of benefits.
This is not a theoretical concern. The thesis identifies three recurring problems in public-sector algorithmic systems: technical opacity, a gap between broad ethical principles and operational practice, and weak institutional protocols for assessing impact before deployment.
In this setting, a black-box system does more than create inconvenience. It can undermine procedural fairness by making it difficult for affected individuals to understand, challenge, or remedy a decision that carries real consequences for their finances, rights, or legal standing.
From Principles to Controls
The contribution of the MEAAT framework (Ethical Framework for AI algorithm in Tax Administration) is that it does not leave ethics at the level of slogans. It translates principles such as transparency, non-discrimination, procedural justice, proportionality, accountability, and security into practical controls that apply across the lifecycle of the system.
The framework includes a Tax Algorithmic Impact Assessment, domain-adapted fairness metrics, tax-specific indices such as IEPT for procedural justice and MPS for punitive proportionality, and public transparency mechanisms through reporting sheets and monitoring dashboards.
This kind of structure matters because legal defensibility is not produced by intention alone. It depends on documented assessments, reviewable criteria, measurable fairness indicators, human oversight, and the capacity to explain why a system acted as it did in a case with real-world effects.
Bias, Scale, and Public Risk
The Amnesty report adds a caution that is especially relevant for governments and public institutions. As generative AI systems scale, they also scale the risks associated with biased training data, discriminatory outputs, and the false perception that bigger systems are necessarily more accurate or trustworthy.
The report also highlights how English-dominant training datasets can marginalize other languages and cultures, producing poor or harmful outputs in contexts that fall outside the system’s dominant linguistic assumptions. In a public-sector environment, that kind of distortion is not merely a quality problem; it can become a rights problem when misclassification or mistranslation affects access, scrutiny, or redress.
This is one reason the thesis warns against weak explainability and insufficient human review in tax settings, particularly when generative AI or large language models are introduced into sensitive workflows. Without robust validation, these systems can produce plausible but unfounded outputs that are difficult to audit and dangerous to operationalize.
Human Oversight Is Not Optional
One of the clearest lessons from both texts is that meaningful human oversight is not a decorative safeguard. It is a core governance requirement for high-impact systems.
In the tax context, this means decisions with serious consequences should not be left to automated scoring alone. Expert review, impact assessment, explainability tools, and auditable decision pathways are necessary if institutions want to preserve both operational effectiveness and public legitimacy.
In the human rights context, the point is even broader. Where a system is built on unlawful or disproportionate data practices, or where its risks of discrimination and rights interference are structurally embedded, the question is not how to optimize deployment but whether deployment should occur at all.
What Defensible AI Looks Like
A defensible AI system is not simply one that performs well on benchmarks. It is one that can show what data it relies on, what risks were identified before deployment, how affected people can contest outcomes, what oversight exists, and what limits have been placed on its use.
That standard is more demanding than the language of innovation usually suggests. But it is also more realistic. In law and public administration, systems are not judged only by output; they are judged by process, accountability, and the fairness of the institutional conditions under which decisions are made.
This is where the intersection between generative AI and tax administration becomes especially revealing. One field exposes the human rights costs of large-scale AI design, while the other shows how those same design failures can become concrete harms when embedded in state decision-making.
So, not every useful technology is acceptable in legal or democratic terms. Artificial intelligence must prove compatibility with rights, due process, and public accountability, not only with efficiency targets or market enthusiasm.
That is why the central test for AI in public life should be simple: not whether it works, but whether it can be defended. If a system cannot explain its basis, justify its effects, and withstand scrutiny from the people it governs, then its usefulness is not enough.
References
Amnesty International. (2026). Unlawful by design: Exposing the human rights costs of generative AI (Index: POL 40/0996/2026).
https://www.amnesty.org
Distefano, Marcela, Ethics in Algorithms Applied to Tax Administrations (December 26, 2025). Available at SSRN: https://ssrn.com/abstract=6811258
Yuppp!! One of the most important shifts in the AI conversation is moving from “Can it do this?” to “Can it justify doing this?” Efficiency without transparency, accountability, and due process is not innovation, it’s a governance risk waiting to surface.
What makes this compelling is that it keeps asking a harder question than “does AI work?” it asks whether these systems can justify the power they’re given. The strongest point is that once AI enters public systems like tax administration, opacity stops being a technical issue and becomes a fairness issue. Efficiency alone is not enough if people cannot understand, challenge, or defend themselves against the system’s decisions.