Evaluability in AI-Mediated Systems When Verification Costs More Than Persuasion

What problem does the paper answer?

What happens to AI-mediated knowledge systems when persuasive informational outputs become inexpensive to produce while independent verification remains costly, demanding, and unevenly distributed?

Evaluability as a governance condition

This paper introduces evaluability as a complementary governance condition: the practical capacity to inspect, challenge, compare, and assess informational claims. It argues that AI-mediated environments increasingly alter the balance between persuasion and verification, allowing confidence in outputs to develop faster than the independent capacities needed to evaluate them.

General-reader summary

This paper examines a practical imbalance created by AI-mediated information systems. AI can now generate coherent, persuasive, and apparently authoritative outputs quickly and cheaply. But verifying those claims still takes time, expertise, access to evidence, comparison across sources, and the ability to reconstruct how particular claims were produced.

The paper argues that this gap creates a governance problem. Existing AI governance discussions often focus on whether outputs are accurate, transparent, explainable, fair, or accountable. Those concerns remain important, but they do not fully answer whether actors outside the system retain meaningful opportunities for independent inspection.

The paper introduces evaluability as the practical capacity to inspect, challenge, compare, and assess informational claims. It then examines how summarization, ranking systems, retrieval infrastructures, and generated synthesis can weaken evaluability by making persuasive outputs easier to accept than to independently verify. The final concern is social as well as technical: evaluative capacity becomes unevenly distributed across institutions and publics.

Manuscript abstract

Artificial intelligence systems increasingly reduce the costs associated with producing persuasive informational outputs while leaving the work of independent verification comparatively resource intensive. Contemporary AI governance discussions have largely focused on characteristics of informational outputs, including accuracy, transparency, explainability, fairness, and accountability. Although these concerns remain essential, they often devote comparatively less attention to whether actors outside the systems generating informational claims retain meaningful opportunities for independent inspection.

This paper introduces evaluability as a complementary governance condition referring to the practical capacity to inspect, challenge, compare, and assess informational claims. It argues that AI-mediated informational environments increasingly alter the relationship between persuasion and verification, creating conditions in which confidence in informational outputs can develop more rapidly than the practical capacities necessary to evaluate them independently. The paper examines several mechanisms contributing to this dynamic, including summarization, ranking systems, retrieval infrastructures, and generated synthesis.

The paper further explores how evaluative capacities become unevenly distributed across institutions and publics. Finally, the paper considers the implications of treating evaluability as a governance objective for sustaining accountability, contestability, public trust, and the resilience of knowledge systems within increasingly AI-mediated informational environments.

Terms to remember

Cheap persuasion changes the knowledge game

This paper matters because it identifies a practical asymmetry at the heart of AI-mediated knowledge systems. When persuasive claims can be generated faster and cheaper than they can be independently checked, trust begins shifting away from inspection and toward plausibility, fluency, institutional authority, or technological confidence.

The mature evaluability paper

This paper consolidates the evaluability sequence into a sharper governance argument. Earlier papers focused on verification preservation, systemic risk, external evaluability, and reconstructability. Paper 21 turns those ideas into a clear practical claim: knowledge systems are vulnerable when verification costs more than persuasion.

Where this paper fits

Journal path

This page preserves the current surviving version associated with Paper 21. The earlier submission used the shorter title When Verification Costs More Than Persuasion.

Nearby work

Knowledge systems become fragile when persuasive outputs become cheaper to produce than they are to independently verify.