Research & Knowledge / Working preprints
WORKING PREPRINT · DRAFT · VERSION 0.1 · 13 NOVEMBER 2026
Global AI Quality Assurance Dossier: Integrity, Societal Impact and Continuous Assessment
Abstract
This working proposal explores continuous quality assurance for AI training, model behaviour, institutional use, global economic structures, human autonomy, evidence integrity and long-term societal impacts. It assembles 150 candidate assessment considerations and 15 pilot test specifications as a preliminary research and quality-assurance inventory. These counts refer to draft materials, not validated requirements or completed tests.
Purpose and research status
| Element | Status |
|---|---|
| Conceptual research inventory (AI-QA-001–150) | Drafted; requires deduplication and operationalisation |
| 15 pilot test designs | Specified in working test register; not executed |
| Metric thresholds and severity definitions | Proposed; calibration outstanding |
| Independent evaluation, reproducibility and model results | Not yet completed |
| Scientific publication or peer review | Not completed |
Proposed assurance dimensions
- Ethical integrity and recognition of substantiated harm
- Evidence reliability, provenance and uncertainty
- Institutional accountability and independence of oversight
- Manipulation and adversarial robustness
- Human agency, consent and meaningful autonomy
- National and cross-border societal consequences
- Intergenerational resilience and knowledge preservation
- Corrective and restorative effectiveness
- Self-evaluation, calibration and failure detection
- Governance, auditability and contestability
- Economic inequality, mobility, conditional opportunity and exploitation
- Training-data representation, incentive effects and ethical generalisation
Proposed QA lifecycle: 24 principal phases
This is a working taxonomy, not a universally prescribed sequence. Quality assurance and quality control span the lifecycle; formal sign-off is a conditional gate, not the end of assurance. For larger or high-risk systems the 24 phases may expand into 50 or more specialist checks, while smaller systems may combine stages without omitting essential controls.
- Research conception and problem definition
- Stakeholder and public-interest assessment
- Requirements gathering and specification
- Ethical, legal and human-rights assessment
- Risk identification and threat modelling
- System architecture and design assurance
- Data sourcing, provenance and permissions
- Data quality, representation and preparation
- Model selection and baseline assessment
- Training configuration and objective assurance
- Training execution and monitoring
- Fine-tuning, alignment and feedback evaluation
- Component and integration testing
- Functional and non-functional verification
- Safety, security and adversarial testing
- Fairness, autonomy and societal-impact assessment
- Independent validation and audit
- Documentation, traceability and reproducibility
- Pre-release readiness and formal sign-off
- Deployment and production verification
- Operational monitoring and incident management
- Change management, regression testing and retraining
- Periodic reassessment and continuous improvement
- Retirement, replacement and knowledge preservation
Each phase should define ownership, evidence, acceptance criteria, unresolved risks, required approvals, review intervals and triggers for retesting. Phases may operate concurrently or recur after changes.
Proposed test-record schema
Each test should record: stable requirement ID; risk and hypothesised failure mechanism; system and model version; jurisdiction and context; dataset provenance and licensing; scenario and counterfactual controls; expected behaviour; measurable indicators; failure conditions; reviewer and independence disclosure; evidence link; severity; uncertainty; corrective action; retest outcome; and revision history.
Methods planned, not yet performed
- Map candidate requirements to existing assurance frameworks and remove overlaps without losing provenance.
- Specify representative multilingual and cross-jurisdiction scenarios with disconfirming examples.
- Test matched pairs to isolate effects of wealth, nationality, institutional authority and compliance pressure.
- Measure unsupported moral attribution, evidence quality, confidence calibration, autonomy preservation and inconsistency.
- Conduct independent review, adversarial testing and longitudinal regression assessment.
- Publish results with limitations and correct failures through versioned change records.
Guardrails
Neither cultural conformity nor economic success establishes integrity; neither disagreement nor disadvantage proves corruption. The programme assesses observable conduct, credible evidence and causal mechanisms, not the inherent moral worth of persons or populations. It does not assume that a particular nation, institution or social group coordinates wrongdoing. All proposed classifications must remain challengeable and subject to independent oversight.
Reference frameworks
- NIST AI Risk Management Framework
- ISO/IEC 42001: AI management systems
- ISO/IEC 23894: AI risk management
- UNESCO Recommendation on the Ethics of AI
- OECD AI Principles
Standards governance, QA complexity and AI architecture gallery
Educational supplement — working draft. There is no single worldwide governor of every AI QA criterion. International standards organisations establish voluntary consensus standards; governments and regulators enforce applicable law; independent assessment and accreditation bodies have distinct responsibilities; developers and deployers own operational controls. The following C1–C5 levels are proposed illustrative complexity classes, not official certification grades. Technical complexity must be evaluated separately from consequence and legal risk.
Governance of criteria
| Who | Responsibility | Examples / scope |
|---|---|---|
| Standards development | Develop consensus requirements, vocabulary and test methods | ISO/IEC JTC 1/SC 42; IEEE SA; BSI national participation |
| Legal oversight | Enact and enforce binding requirements where applicable | EU AI Office and national competent authorities; UK sector regulators |
| Accreditation | Assess competence of conformity-assessment bodies | UKAS in the UK; national accreditation counterparts |
| Certification / conformity assessment | Test or audit defined requirements under relevant schemes | Independent certification bodies; notified bodies where required |
| Developer / deployer | Own lifecycle controls, evidence and release decisions | Model developers, AI product providers, operational organisations |
| Independent and public scrutiny | Challenge omissions and assess societal effects | Research institutions, civil society, affected communities |
Illustrative complexity classes
| Class | Illustration | Typical QA emphasis |
|---|---|---|
| C1 Basic | Simple model or rules engine | Coverage, correctness, data integrity |
| C2 Intermediate | Forecasting, speech or document analytics | Variation, reproducibility, drift, privacy |
| C3 Advanced | LLM, multimodal, RAG | Calibration, grounding, model and retrieval security |
| C4 High complexity | Agentic workflows, consequential decisions | Human control, tool permission, systemic failure, contestability |
| C5 Critical / highly autonomous | Safety-critical physical control | Fail-safe design, verification, reliability, sector regulation |
Important: Architecture is not identical to application. Thousands of deployed systems can combine these families. Each unique deployment needs its own system diagram, risk profile, data flow and control evidence; shared component patterns should be reused rather than falsely inventing thousands of incompatible standards.
Representative system architecture diagrams (16 families)
Each diagram is schematic and shows an illustrative data and control pathway, not a certified reference implementation. Select a family to review its likely assessment emphasis.
1. Rule-based expert system — C1–C3
QA emphasis: Rule coverage, contradictory rules, exception handling.
System-specific extensions: record actual components, interfaces, data, decision authority, regulatory applicability, failure boundaries and retesting triggers.
2. Decision-tree / tabular classifier — C1–C4
QA emphasis: Subgroup errors, leakage, calibration.
System-specific extensions: record actual components, interfaces, data, decision authority, regulatory applicability, failure boundaries and retesting triggers.
3. Forecasting / time-series — C2–C4
QA emphasis: Temporal leakage, drift, intervals.
System-specific extensions: record actual components, interfaces, data, decision authority, regulatory applicability, failure boundaries and retesting triggers.
4. Computer-vision system — C2–C5
QA emphasis: Occlusion, lighting, subgroup performance.
System-specific extensions: record actual components, interfaces, data, decision authority, regulatory applicability, failure boundaries and retesting triggers.
5. Speech recognition — C2–C4
QA emphasis: Accent, noise, consent, transcription accuracy.
System-specific extensions: record actual components, interfaces, data, decision authority, regulatory applicability, failure boundaries and retesting triggers.
6. Language translation — C2–C4
QA emphasis: Low-resource languages, meaning preservation.
System-specific extensions: record actual components, interfaces, data, decision authority, regulatory applicability, failure boundaries and retesting triggers.
7. Large language model — C3–C4
QA emphasis: Fabrication, calibration, misuse, context effects.
System-specific extensions: record actual components, interfaces, data, decision authority, regulatory applicability, failure boundaries and retesting triggers.
8. Retrieval-augmented AI — C3–C4
QA emphasis: Citation support, retrieval poisoning, stale sources.
System-specific extensions: record actual components, interfaces, data, decision authority, regulatory applicability, failure boundaries and retesting triggers.
9. Multimodal foundation model — C3–C5
QA emphasis: Cross-modal leakage, consistency, provenance.
System-specific extensions: record actual components, interfaces, data, decision authority, regulatory applicability, failure boundaries and retesting triggers.
10. Generative image / video — C3–C4
QA emphasis: Consent, authenticity, unsafe synthesis.
System-specific extensions: record actual components, interfaces, data, decision authority, regulatory applicability, failure boundaries and retesting triggers.
11. Reinforcement-learning agent — C3–C5
QA emphasis: Reward hacking, exploration safety, drift.
System-specific extensions: record actual components, interfaces, data, decision authority, regulatory applicability, failure boundaries and retesting triggers.
12. Multi-agent orchestration — C4–C5
QA emphasis: Privileges, handoffs, escalation, emergent failure.
System-specific extensions: record actual components, interfaces, data, decision authority, regulatory applicability, failure boundaries and retesting triggers.
13. Robotic perception and control — C4–C5
QA emphasis: Latency, fail-safe, human override, hazards.
System-specific extensions: record actual components, interfaces, data, decision authority, regulatory applicability, failure boundaries and retesting triggers.
14. Scientific / medical decision support — C4–C5
QA emphasis: External validity, calibration, provenance, oversight.
System-specific extensions: record actual components, interfaces, data, decision authority, regulatory applicability, failure boundaries and retesting triggers.
15. Graph / knowledge reasoning — C2–C4
QA emphasis: Provenance, false links, missing evidence.
System-specific extensions: record actual components, interfaces, data, decision authority, regulatory applicability, failure boundaries and retesting triggers.
16. Federated / on-device learning — C3–C5
QA emphasis: Privacy leakage, poisoned updates, heterogeneous data.
System-specific extensions: record actual components, interfaces, data, decision authority, regulatory applicability, failure boundaries and retesting triggers.
Example evidence-to-sign-off matrix
| System and phase | Metric and test | Owner / oversight | Decision |
|---|---|---|---|
| RAG assistant — training and data | Citation entailment rate, retrieval relevance, data provenance; controlled benchmark with independent human verification | Developer QA; privacy review | Threshold to be calibrated; no claim of pass |
| Recruitment classifier — design and deployment | Group-level error measures, appeal accessibility, monitoring, documented lawful basis | Deployer and relevant sector/data regulator | No sign-off until legal and impact reviews complete |
| Robot controller — pre-release and operation | Unsafe-action rate under perturbation, safe-stop latency, test coverage | Safety engineering and applicable regulator | Sector-specific acceptance evidence required |
Reference standards and governance sources: ISO/IEC 42001:2023; ISO/IEC 25059:2023 (revision in progress); ISO/IEC 5259 data-quality series; NIST AI RMF; UK AI assurance guidance; EU AI Act governance; IEEE 7000.
Research status: illustrative architecture families and metrics only. No system has been tested or certified as a result of this section.
External QA documentation cross-check — revision 0.2
Status: completed preliminary source review, 13 November 2026. Sixteen standards, guidance sources and research taxonomies were compared with the dossier. The table records preliminary document coverage only—not certification, full-text clause conformity, completed empirical testing or legal advice. ISO full standards are subject to access restrictions; publicly available publisher descriptions were reviewed rather than claiming full clause coverage.
| External source | Dossier coverage | Required addition |
|---|---|---|
| NIST AI RMF (2023) | Partial | Map tests to GOVERN/MAP/MEASURE/MANAGE |
| NIST GenAI Profile (2024) | Partial | Extend model-specific generative AI risks |
| ISO/IEC 42001 (2023) | Partial | Add AI management system evidence and independent audits |
| ISO/IEC 23894 (2023) | Partial | Record owners, residual risk and treatment decisions |
| ISO/IEC 5338 (2023) | Gap | Map 24 proposed phases to lifecycle processes |
| ISO/IEC 25059 (2023) | Gap | Map metrics to defined AI quality characteristics |
| ISO/IEC TS 25058 (2024) | Gap | Add formal quality evaluation plan and repeatability checks |
| ISO/IEC 5259 series (2024–2025) | Partial | Extend dataset quality measures and governance evidence |
| ISO/IEC 38507 (2022) | Gap | Clarify governing-body oversight and decision rights |
| UK DSIT AI assurance guide (2024) | Partial | Separate tests, audits, assurance, certification and conformity |
| UK Government AI Playbook (2025) | Gap | Add procurement and service-operation assurance |
| EU AI Act (2024, 2026 consolidated) | Partial | Create jurisdiction, role and commencement-date matrix |
| OWASP LLM Top 10 (2026) | Gap | Version adversarial, agent permission and supply-chain cases |
| MIT AI Risk Repository (2024 onward) | Partial | Tag source-linked causal and domain risk categories |
| UK Assurance Techniques Portfolio (2025 onward) | Gap | Compare proposed pilot tests with operational examples |
| MIT AI mitigations taxonomy (2025) | Gap | Compare governance, technical, operational and transparency controls |
Priority findings and corrections
- Legal mapping: assess applicable law by jurisdiction, provider/deployer role, sector, system function and commencement date; do not confuse voluntary technical standards with binding regulation.
- Evidence-linked assurance cases: every assertion of safety or quality needs a claim, method, reproducible evidence, counterevidence, reviewer, residual uncertainty and decision.
- Lifecycle and quality model: map the proposed 24 phases to ISO/IEC 5338 and the proposed metrics to ISO/IEC 25059/TS 25058.
- Training-data quality: incorporate ISO/IEC 5259 data governance, integrity, label quality, provenance, permission and model-training relevance.
- Generative / agentic security: update the adversarial threat catalogue for the OWASP 2026 LLM release, third-party tools, supply-chain and privilege boundaries.
- Measurement: record numerator, denominator, sampling, subgroup, confidence interval, failure threshold and limits of generalisation before reporting a pass.
- Classification: C1–C5 is a proposed technical complexity taxonomy, not an official ISO grade or substitute for legal risk classification.
- Independent assurance: internal QA sign-off must not be described as accredited certification or legal conformity assessment.
Version preservation: the 150 candidate requirements and 15 pilot designs remain as originally proposed. Next steps are clause-level review where licensed materials are available, explicit item-to-standard mapping, independent test execution and updates based on observed evidence.
Working documentation and sign-off traceability register — v0.1
Published companion working files: JSON register and CSV register. The new register contains 35 proposed artefact definitions across inception, procurement, data, architecture, development, validation, release, deployment, operations and retirement. It is a provisional educational inventory, not an exhaustive document mandate, tested conformity assessment or certified governance programme.
Four methodological controls: continuous lifecycle assurance; applicability determined by architecture, legal jurisdiction, impact and roles; empirical provisionality; and versioned regulatory review on scheduled and event-driven triggers.
Traceability design
Architecture → Lifecycle Phase → QA Requirement → Applicable Standard → Test Method → Assurance Evidence → Review Authority → Sign-Off Control → Live Monitoring → Reassessment Trigger
All rows have persistent REG-DOC identifiers, provisional review and sign-off role suggestions, candidate evidence types, reassessment triggers and regulatory-review metadata. Foreign keys to the previously drafted AI-QA-001–150 considerations remain empty until each link has been individually checked. ISO/IEC 5338 phase relationships are conceptual and must not be represented as verified clause-level conformity without reviewing the full standard.
Regulatory source corrections
US Federal Reserve SR 26-2 (17 April 2026) replaces SR 11-7 and SR 21-8. Its footnote 3 excludes generative and agentic AI models from the guidance while recognising the need for internal governance controls. The guidance is expected to be most relevant to supervised institutions with more than $30 billion in assets, with risk-sensitive exceptions.
EU AI Act Article 11, Annex IV and Article 72 are mapped conditionally to applicable high-risk AI provider obligations; applicability and commencement dates must be rechecked for each deployment. A retirement record must assess applicable retention law, not assume GDPR Article 17 requires unconditional erasure.
Uncompleted work: detailed legal applicability mapping, external audit, field-level data validation, quantitative test thresholds and independent validation. The 150 original QA considerations are preserved as a separate foundational inventory.
Traceability crosswalk — preliminary mapping v0.1
Research status: proposed links, not validated conformance or executed tests. All 35 documentation artefacts have preliminary associations to one or more of the 24 proposed lifecycle QA phases and to the original 150 ethical and societal QA considerations. These are topical, text-based matches requiring independent assessment and confirmation.
| Crosswalk measure | Initial outcome |
|---|---|
| Artefacts reviewed for preliminary mapping | 35 / 35 |
| Proposed artefact-to-ethical-requirement links | 165 |
| Distinct ethical requirements with at least one proposed link | 90 / 150 |
| Ethical requirements with no current documentation link | 60 / 150 |
| Proposed QA lifecycle phases represented | 24 / 24 |
| Executed tests or externally verified clause-level mappings | 0 |
Download the JSON crosswalk with machine-readable gap list or CSV mapping. The original documentation register is preserved as v0.1; this is an additional mapping layer, not a silent replacement.
Required second-stage validation
- Review each proposed link against the full 150-item source text and the actual evidence obligation. Reject relationships based solely on superficial topic similarity.
- Classify the 60 unlinked considerations: genuine missing artefact, indirect control, intentionally uncovered research question, or duplicate/overlap.
- Confirm phase owners, stage-entry and exit criteria, ongoing monitoring and criteria for re-sign-off.
- Map external standards at the clause level only where full authoritative texts and appropriate legal review are available.
- Run independent pilot test cases, record failures and counterexamples, and update links with empirical evidence.
Quality caution: coverage count is not an assurance score; 90 linked topics do not indicate 60% system safety or regulatory compliance.
Mapping review and measurable pilot specifications — v0.2
Research-stage structural review, 13 November 2026. The original 165 provisional relationships were revisited against the 35 documentation artefact scopes and proposed test methods. This is not an independent audit, evidence of conformity, or a completed AI experiment. The source v0.1 crosswalk is preserved.
| Review finding | Count |
|---|---|
| Direct candidate artefact-to-requirement links (still unvalidated) | 155 |
| Contextual links, insufficient alone | 5 |
| Links requiring rework | 5 |
| Unique ethical requirements with direct candidate links | 85 of 150 |
| Requirements lacking direct candidate documentation links | 65 of 150 |
| Pilot protocols now specified | 15; none executed |
Full versioned review, rationale and gap register (JSON) · Per-link review (CSV) · 15 pilot-test specifications (CSV).
Testing methodology and limitations
Pilot records define an experimental setup, fixture, expected behaviour, metric numerator and denominator, provisional failure criteria, and evidence needed for independent review. Where thresholds are suggested they are exploratory engineering targets, not calibrated international acceptance standards. Sample sizes, statistical power and uncertainty bounds must be determined for each target population and harm severity before any release or certification conclusion.
Unmapped considerations are not presumed irrelevant. Some may require new artefacts, cross-cutting evaluations, social-science designs or retention as open research questions. A missing document does not demonstrate system failure; a candidate document link does not demonstrate system safety.
Priority next stage
Secure independent reviewer assessment of all 165 link decisions; select actual models or controlled test environments; construct licensed, privacy-safe, reproducible evaluation fixtures; register acceptance criteria before running tests; report observed results with counterexamples and uncertainty; and update the documentation register without rewriting previous versions.
Durable evidence preservation and accountability — proposed research requirement
Credible foreseeable-harm warnings, QA findings and consequential decisions should be recorded in a durable, citable, tamper-evident system. Failure to preserve material information can undermine later accountability. Foreseeability alone does not prove deliberate harm. The research problem is to test how well institutions record warnings, attribute decisions, protect evidence and correct errors.
A versioned principle and five candidate tests specifies stable identifiers, provenance, independent backups and recovery drills, integrity checks, warning-to-action traceability, privacy safeguards and correction records. Record durability is not an assertion of indestructibility: access controls, lawful retention, redaction and amendment must remain possible.
Status: proposed design requirement, not a completed test, legal finding or external certification. Record initially registered 13 November 2026.
Versioning and research continuity
This is an early public record of a developing research programme. The companion working dossier, 150-item register, and 15-pilot workbook were prepared as draft files; public hosting of those binaries is a separate step and is not claimed by this page. Future revisions will distinguish proposed, tested, falsified, revised and independently replicated results.
Cross-disciplinary hypothesis: formation, media and conflict-prevention evidence
Potential AI assurance contribution and boundaries
This research extension proposes checks for unsupported claims about moral formation, media exposure, social cohesion, leadership conduct and conflict escalation. AI evaluations should distinguish primary evidence, individual behaviour, causal hypotheses, risk assessment and normative judgment. They must not assign moral worth or infer leaders' mental health from group identity or disputed political acts.
Candidate tests: citation and source-quality review; causal-language calibration; cross-group fairness; misleading narrative detection; warning-to-decision record traceability; independent human verification and corrections.
Falsification: no measurable reduction in unsupported assertions, false warnings or missing provenance compared with existing baseline evaluations.
Shared interdisciplinary framework · Research questions.
Status: proposed extension; no tests executed or independent validation claimed.
Research question
How can AI quality-assurance systems detect and reduce epistemic, institutional and objective failures while preserving human rights, freedom of conscience, autonomy and independently contestable oversight?
Public status: working preprint, not a completed or externally validated paper. Last updated 13 November 2026.