INDEPENDENT RESEARCH ยท PRACTICAL EXPLANATION
How to audit AI systems for bias, reliability and explainability
An AI audit needs a defined system, intended use, evidence and evaluation criteria. This guide describes a research-oriented workflow for identifying risks and reporting limitations; it is not a certification or a claim that any specific system has been audited.
Scope: Informational research guide. Adapt the method to the relevant domain; follow applicable professional, legal and safeguarding requirements.
1. Define the audit scope
Record system version, intended users, permitted tasks, deployment setting, data sources, known constraints and decisions the system may influence. Specify what the audit does not cover.
2. Choose measures appropriate to the task
Depending on context, measure factual accuracy, false-positive and false-negative rates, calibration, latency, consistency, human override and error severity. No single metric establishes overall fairness.
3. Build representative test sets
Test common, uncommon and difficult cases; document sampling limitations. Where lawful and ethical, evaluate relevant subgroup outcomes while avoiding inappropriate inference about protected characteristics.
4. Check robustness and explainability
Vary phrasing, data quality and contextual inputs; compare outputs under meaningful changes. Explainability requires usable evidence about a system's behaviour, not merely confident-sounding explanations.
5. Record harms and human oversight
Document incidents, foreseeable misuse, escalation thresholds, responsibilities, challenge routes and mechanisms for correction. Distinguish research observations from organisational decisions.
6. Publish auditable findings
Provide method, test dates, model version, denominator, uncertainty, failures, limitations, mitigation actions and a decision log. A model card or risk register can support traceability.
Worked reasoning example
Example: For an information-retrieval assistant, compare answer accuracy on a documented set of routine and edge-case prompts, track unsupported citations and corrections, then report the conditions under which performance degrades.
Related Manifold Grace research
Standards and supporting references
Published 9 November 2026. This page describes a methodology; it is not a record of a completed external audit or a guarantee of search placement.