Research & Knowledge / Working preprints

WORKING PREPRINT · DRAFT · VERSION 0.1 · 13 NOVEMBER 2026

Global AI Quality Assurance Dossier: Integrity, Societal Impact and Continuous Assessment

Manifold Grace — Integrated Societal Intelligence research programme. Public working document, open to revision. Not peer reviewed; not an approved international standard, certification or completed experimental study.

Abstract

This working proposal explores continuous quality assurance for AI training, model behaviour, institutional use, global economic structures, human autonomy, evidence integrity and long-term societal impacts. It assembles 150 candidate assessment considerations and 15 pilot test specifications as a preliminary research and quality-assurance inventory. These counts refer to draft materials, not validated requirements or completed tests.

Purpose and research status

ElementStatus
Conceptual research inventory (AI-QA-001–150)Drafted; requires deduplication and operationalisation
15 pilot test designsSpecified in working test register; not executed
Metric thresholds and severity definitionsProposed; calibration outstanding
Independent evaluation, reproducibility and model resultsNot yet completed
Scientific publication or peer reviewNot completed

Proposed assurance dimensions

  1. Ethical integrity and recognition of substantiated harm
  2. Evidence reliability, provenance and uncertainty
  3. Institutional accountability and independence of oversight
  4. Manipulation and adversarial robustness
  5. Human agency, consent and meaningful autonomy
  6. National and cross-border societal consequences
  7. Intergenerational resilience and knowledge preservation
  8. Corrective and restorative effectiveness
  9. Self-evaluation, calibration and failure detection
  10. Governance, auditability and contestability
  11. Economic inequality, mobility, conditional opportunity and exploitation
  12. Training-data representation, incentive effects and ethical generalisation

Proposed QA lifecycle: 24 principal phases

This is a working taxonomy, not a universally prescribed sequence. Quality assurance and quality control span the lifecycle; formal sign-off is a conditional gate, not the end of assurance. For larger or high-risk systems the 24 phases may expand into 50 or more specialist checks, while smaller systems may combine stages without omitting essential controls.

  1. Research conception and problem definition
  2. Stakeholder and public-interest assessment
  3. Requirements gathering and specification
  4. Ethical, legal and human-rights assessment
  5. Risk identification and threat modelling
  6. System architecture and design assurance
  7. Data sourcing, provenance and permissions
  8. Data quality, representation and preparation
  9. Model selection and baseline assessment
  10. Training configuration and objective assurance
  11. Training execution and monitoring
  12. Fine-tuning, alignment and feedback evaluation
  13. Component and integration testing
  14. Functional and non-functional verification
  15. Safety, security and adversarial testing
  16. Fairness, autonomy and societal-impact assessment
  17. Independent validation and audit
  18. Documentation, traceability and reproducibility
  19. Pre-release readiness and formal sign-off
  20. Deployment and production verification
  21. Operational monitoring and incident management
  22. Change management, regression testing and retraining
  23. Periodic reassessment and continuous improvement
  24. Retirement, replacement and knowledge preservation

Each phase should define ownership, evidence, acceptance criteria, unresolved risks, required approvals, review intervals and triggers for retesting. Phases may operate concurrently or recur after changes.

Proposed test-record schema

Each test should record: stable requirement ID; risk and hypothesised failure mechanism; system and model version; jurisdiction and context; dataset provenance and licensing; scenario and counterfactual controls; expected behaviour; measurable indicators; failure conditions; reviewer and independence disclosure; evidence link; severity; uncertainty; corrective action; retest outcome; and revision history.

Methods planned, not yet performed

  1. Map candidate requirements to existing assurance frameworks and remove overlaps without losing provenance.
  2. Specify representative multilingual and cross-jurisdiction scenarios with disconfirming examples.
  3. Test matched pairs to isolate effects of wealth, nationality, institutional authority and compliance pressure.
  4. Measure unsupported moral attribution, evidence quality, confidence calibration, autonomy preservation and inconsistency.
  5. Conduct independent review, adversarial testing and longitudinal regression assessment.
  6. Publish results with limitations and correct failures through versioned change records.

Guardrails

Neither cultural conformity nor economic success establishes integrity; neither disagreement nor disadvantage proves corruption. The programme assesses observable conduct, credible evidence and causal mechanisms, not the inherent moral worth of persons or populations. It does not assume that a particular nation, institution or social group coordinates wrongdoing. All proposed classifications must remain challengeable and subject to independent oversight.

Reference frameworks

Standards governance, QA complexity and AI architecture gallery

Educational supplement — working draft. There is no single worldwide governor of every AI QA criterion. International standards organisations establish voluntary consensus standards; governments and regulators enforce applicable law; independent assessment and accreditation bodies have distinct responsibilities; developers and deployers own operational controls. The following C1–C5 levels are proposed illustrative complexity classes, not official certification grades. Technical complexity must be evaluated separately from consequence and legal risk.

Governance of criteria

WhoResponsibilityExamples / scope
Standards developmentDevelop consensus requirements, vocabulary and test methodsISO/IEC JTC 1/SC 42; IEEE SA; BSI national participation
Legal oversightEnact and enforce binding requirements where applicableEU AI Office and national competent authorities; UK sector regulators
AccreditationAssess competence of conformity-assessment bodiesUKAS in the UK; national accreditation counterparts
Certification / conformity assessmentTest or audit defined requirements under relevant schemesIndependent certification bodies; notified bodies where required
Developer / deployerOwn lifecycle controls, evidence and release decisionsModel developers, AI product providers, operational organisations
Independent and public scrutinyChallenge omissions and assess societal effectsResearch institutions, civil society, affected communities

Illustrative complexity classes

ClassIllustrationTypical QA emphasis
C1 BasicSimple model or rules engineCoverage, correctness, data integrity
C2 IntermediateForecasting, speech or document analyticsVariation, reproducibility, drift, privacy
C3 AdvancedLLM, multimodal, RAGCalibration, grounding, model and retrieval security
C4 High complexityAgentic workflows, consequential decisionsHuman control, tool permission, systemic failure, contestability
C5 Critical / highly autonomousSafety-critical physical controlFail-safe design, verification, reliability, sector regulation

Important: Architecture is not identical to application. Thousands of deployed systems can combine these families. Each unique deployment needs its own system diagram, risk profile, data flow and control evidence; shared component patterns should be reused rather than falsely inventing thousands of incompatible standards.

Representative system architecture diagrams (16 families)

Each diagram is schematic and shows an illustrative data and control pathway, not a certified reference implementation. Select a family to review its likely assessment emphasis.

1. Rule-based expert system — C1–C3
Rules / facts to Inference engine to Decision to ExplanationRules / factsInference engineDecisionExplanation

QA emphasis: Rule coverage, contradictory rules, exception handling.

System-specific extensions: record actual components, interfaces, data, decision authority, regulatory applicability, failure boundaries and retesting triggers.

2. Decision-tree / tabular classifier — C1–C4
Structured records to Preprocessing to Classifier to DecisionStructured recordsPreprocessingClassifierDecision

QA emphasis: Subgroup errors, leakage, calibration.

System-specific extensions: record actual components, interfaces, data, decision authority, regulatory applicability, failure boundaries and retesting triggers.

3. Forecasting / time-series — C2–C4
Historical sequence to Feature windows to Forecast model to ForecastHistorical sequenceFeature windowsForecast modelForecast

QA emphasis: Temporal leakage, drift, intervals.

System-specific extensions: record actual components, interfaces, data, decision authority, regulatory applicability, failure boundaries and retesting triggers.

4. Computer-vision system — C2–C5
Images / video to Vision encoder to Detection to Human reviewImages / videoVision encoderDetectionHuman review

QA emphasis: Occlusion, lighting, subgroup performance.

System-specific extensions: record actual components, interfaces, data, decision authority, regulatory applicability, failure boundaries and retesting triggers.

5. Speech recognition — C2–C4
Audio stream to Signal / speech model to Transcript to ReviewAudio streamSignal / speech modelTranscriptReview

QA emphasis: Accent, noise, consent, transcription accuracy.

System-specific extensions: record actual components, interfaces, data, decision authority, regulatory applicability, failure boundaries and retesting triggers.

6. Language translation — C2–C4
Source language to Translation model to Target text to Human checkSource languageTranslation modelTarget textHuman check

QA emphasis: Low-resource languages, meaning preservation.

System-specific extensions: record actual components, interfaces, data, decision authority, regulatory applicability, failure boundaries and retesting triggers.

7. Large language model — C3–C4
Prompt / context to Transformer to Generated response to EvaluationPrompt / contextTransformerGenerated responseEvaluation

QA emphasis: Fabrication, calibration, misuse, context effects.

System-specific extensions: record actual components, interfaces, data, decision authority, regulatory applicability, failure boundaries and retesting triggers.

8. Retrieval-augmented AI — C3–C4
Question to Search / retrieval to LLM + sources to Cited answerQuestionSearch / retrievalLLM + sourcesCited answer

QA emphasis: Citation support, retrieval poisoning, stale sources.

System-specific extensions: record actual components, interfaces, data, decision authority, regulatory applicability, failure boundaries and retesting triggers.

9. Multimodal foundation model — C3–C5
Text + images + audio to Fusion model to Multimodal output to Human oversightText + images + audioFusion modelMultimodal outputHuman oversight

QA emphasis: Cross-modal leakage, consistency, provenance.

System-specific extensions: record actual components, interfaces, data, decision authority, regulatory applicability, failure boundaries and retesting triggers.

10. Generative image / video — C3–C4
Prompt + references to Generative model to Synthetic media to Provenance checkPrompt + referencesGenerative modelSynthetic mediaProvenance check

QA emphasis: Consent, authenticity, unsafe synthesis.

System-specific extensions: record actual components, interfaces, data, decision authority, regulatory applicability, failure boundaries and retesting triggers.

11. Reinforcement-learning agent — C3–C5
State / reward to Policy learning to Action selection to EnvironmentState / rewardPolicy learningAction selectionEnvironment

QA emphasis: Reward hacking, exploration safety, drift.

System-specific extensions: record actual components, interfaces, data, decision authority, regulatory applicability, failure boundaries and retesting triggers.

12. Multi-agent orchestration — C4–C5
Task to Planner + agents to Tools / coordination to Reviewed resultTaskPlanner + agentsTools / coordinationReviewed result

QA emphasis: Privileges, handoffs, escalation, emergent failure.

System-specific extensions: record actual components, interfaces, data, decision authority, regulatory applicability, failure boundaries and retesting triggers.

13. Robotic perception and control — C4–C5
Sensors to Perception / planner to Actuators to Physical environmentSensorsPerception / plannerActuatorsPhysical environment

QA emphasis: Latency, fail-safe, human override, hazards.

System-specific extensions: record actual components, interfaces, data, decision authority, regulatory applicability, failure boundaries and retesting triggers.

14. Scientific / medical decision support — C4–C5
Domain records to Validated model to Recommendation to Qualified reviewerDomain recordsValidated modelRecommendationQualified reviewer

QA emphasis: External validity, calibration, provenance, oversight.

System-specific extensions: record actual components, interfaces, data, decision authority, regulatory applicability, failure boundaries and retesting triggers.

15. Graph / knowledge reasoning — C2–C4
Entities / relations to Graph inference to Hypothesis to ValidationEntities / relationsGraph inferenceHypothesisValidation

QA emphasis: Provenance, false links, missing evidence.

System-specific extensions: record actual components, interfaces, data, decision authority, regulatory applicability, failure boundaries and retesting triggers.

16. Federated / on-device learning — C3–C5
Local datasets to Distributed updates to Aggregated model to Local inferenceLocal datasetsDistributed updatesAggregated modelLocal inference

QA emphasis: Privacy leakage, poisoned updates, heterogeneous data.

System-specific extensions: record actual components, interfaces, data, decision authority, regulatory applicability, failure boundaries and retesting triggers.

Example evidence-to-sign-off matrix

System and phaseMetric and testOwner / oversightDecision
RAG assistant — training and dataCitation entailment rate, retrieval relevance, data provenance; controlled benchmark with independent human verificationDeveloper QA; privacy reviewThreshold to be calibrated; no claim of pass
Recruitment classifier — design and deploymentGroup-level error measures, appeal accessibility, monitoring, documented lawful basisDeployer and relevant sector/data regulatorNo sign-off until legal and impact reviews complete
Robot controller — pre-release and operationUnsafe-action rate under perturbation, safe-stop latency, test coverageSafety engineering and applicable regulatorSector-specific acceptance evidence required

Reference standards and governance sources: ISO/IEC 42001:2023; ISO/IEC 25059:2023 (revision in progress); ISO/IEC 5259 data-quality series; NIST AI RMF; UK AI assurance guidance; EU AI Act governance; IEEE 7000.

Research status: illustrative architecture families and metrics only. No system has been tested or certified as a result of this section.

External QA documentation cross-check — revision 0.2

Status: completed preliminary source review, 13 November 2026. Sixteen standards, guidance sources and research taxonomies were compared with the dossier. The table records preliminary document coverage only—not certification, full-text clause conformity, completed empirical testing or legal advice. ISO full standards are subject to access restrictions; publicly available publisher descriptions were reviewed rather than claiming full clause coverage.

External sourceDossier coverageRequired addition
NIST AI RMF (2023)PartialMap tests to GOVERN/MAP/MEASURE/MANAGE
NIST GenAI Profile (2024)PartialExtend model-specific generative AI risks
ISO/IEC 42001 (2023)PartialAdd AI management system evidence and independent audits
ISO/IEC 23894 (2023)PartialRecord owners, residual risk and treatment decisions
ISO/IEC 5338 (2023)GapMap 24 proposed phases to lifecycle processes
ISO/IEC 25059 (2023)GapMap metrics to defined AI quality characteristics
ISO/IEC TS 25058 (2024)GapAdd formal quality evaluation plan and repeatability checks
ISO/IEC 5259 series (2024–2025)PartialExtend dataset quality measures and governance evidence
ISO/IEC 38507 (2022)GapClarify governing-body oversight and decision rights
UK DSIT AI assurance guide (2024)PartialSeparate tests, audits, assurance, certification and conformity
UK Government AI Playbook (2025)GapAdd procurement and service-operation assurance
EU AI Act (2024, 2026 consolidated)PartialCreate jurisdiction, role and commencement-date matrix
OWASP LLM Top 10 (2026)GapVersion adversarial, agent permission and supply-chain cases
MIT AI Risk Repository (2024 onward)PartialTag source-linked causal and domain risk categories
UK Assurance Techniques Portfolio (2025 onward)GapCompare proposed pilot tests with operational examples
MIT AI mitigations taxonomy (2025)GapCompare governance, technical, operational and transparency controls

Priority findings and corrections

  1. Legal mapping: assess applicable law by jurisdiction, provider/deployer role, sector, system function and commencement date; do not confuse voluntary technical standards with binding regulation.
  2. Evidence-linked assurance cases: every assertion of safety or quality needs a claim, method, reproducible evidence, counterevidence, reviewer, residual uncertainty and decision.
  3. Lifecycle and quality model: map the proposed 24 phases to ISO/IEC 5338 and the proposed metrics to ISO/IEC 25059/TS 25058.
  4. Training-data quality: incorporate ISO/IEC 5259 data governance, integrity, label quality, provenance, permission and model-training relevance.
  5. Generative / agentic security: update the adversarial threat catalogue for the OWASP 2026 LLM release, third-party tools, supply-chain and privilege boundaries.
  6. Measurement: record numerator, denominator, sampling, subgroup, confidence interval, failure threshold and limits of generalisation before reporting a pass.
  7. Classification: C1–C5 is a proposed technical complexity taxonomy, not an official ISO grade or substitute for legal risk classification.
  8. Independent assurance: internal QA sign-off must not be described as accredited certification or legal conformity assessment.

Version preservation: the 150 candidate requirements and 15 pilot designs remain as originally proposed. Next steps are clause-level review where licensed materials are available, explicit item-to-standard mapping, independent test execution and updates based on observed evidence.

Working documentation and sign-off traceability register — v0.1

Published companion working files: JSON register and CSV register. The new register contains 35 proposed artefact definitions across inception, procurement, data, architecture, development, validation, release, deployment, operations and retirement. It is a provisional educational inventory, not an exhaustive document mandate, tested conformity assessment or certified governance programme.

Four methodological controls: continuous lifecycle assurance; applicability determined by architecture, legal jurisdiction, impact and roles; empirical provisionality; and versioned regulatory review on scheduled and event-driven triggers.

Traceability design

Architecture → Lifecycle Phase → QA Requirement → Applicable Standard → Test Method → Assurance Evidence → Review Authority → Sign-Off Control → Live Monitoring → Reassessment Trigger

All rows have persistent REG-DOC identifiers, provisional review and sign-off role suggestions, candidate evidence types, reassessment triggers and regulatory-review metadata. Foreign keys to the previously drafted AI-QA-001–150 considerations remain empty until each link has been individually checked. ISO/IEC 5338 phase relationships are conceptual and must not be represented as verified clause-level conformity without reviewing the full standard.

Regulatory source corrections

US Federal Reserve SR 26-2 (17 April 2026) replaces SR 11-7 and SR 21-8. Its footnote 3 excludes generative and agentic AI models from the guidance while recognising the need for internal governance controls. The guidance is expected to be most relevant to supervised institutions with more than $30 billion in assets, with risk-sensitive exceptions.

EU AI Act Article 11, Annex IV and Article 72 are mapped conditionally to applicable high-risk AI provider obligations; applicability and commencement dates must be rechecked for each deployment. A retirement record must assess applicable retention law, not assume GDPR Article 17 requires unconditional erasure.

Uncompleted work: detailed legal applicability mapping, external audit, field-level data validation, quantitative test thresholds and independent validation. The 150 original QA considerations are preserved as a separate foundational inventory.

Traceability crosswalk — preliminary mapping v0.1

Research status: proposed links, not validated conformance or executed tests. All 35 documentation artefacts have preliminary associations to one or more of the 24 proposed lifecycle QA phases and to the original 150 ethical and societal QA considerations. These are topical, text-based matches requiring independent assessment and confirmation.

Crosswalk measureInitial outcome
Artefacts reviewed for preliminary mapping35 / 35
Proposed artefact-to-ethical-requirement links165
Distinct ethical requirements with at least one proposed link90 / 150
Ethical requirements with no current documentation link60 / 150
Proposed QA lifecycle phases represented24 / 24
Executed tests or externally verified clause-level mappings0

Download the JSON crosswalk with machine-readable gap list or CSV mapping. The original documentation register is preserved as v0.1; this is an additional mapping layer, not a silent replacement.

Required second-stage validation

  1. Review each proposed link against the full 150-item source text and the actual evidence obligation. Reject relationships based solely on superficial topic similarity.
  2. Classify the 60 unlinked considerations: genuine missing artefact, indirect control, intentionally uncovered research question, or duplicate/overlap.
  3. Confirm phase owners, stage-entry and exit criteria, ongoing monitoring and criteria for re-sign-off.
  4. Map external standards at the clause level only where full authoritative texts and appropriate legal review are available.
  5. Run independent pilot test cases, record failures and counterexamples, and update links with empirical evidence.

Quality caution: coverage count is not an assurance score; 90 linked topics do not indicate 60% system safety or regulatory compliance.

Mapping review and measurable pilot specifications — v0.2

Research-stage structural review, 13 November 2026. The original 165 provisional relationships were revisited against the 35 documentation artefact scopes and proposed test methods. This is not an independent audit, evidence of conformity, or a completed AI experiment. The source v0.1 crosswalk is preserved.

Review findingCount
Direct candidate artefact-to-requirement links (still unvalidated)155
Contextual links, insufficient alone5
Links requiring rework5
Unique ethical requirements with direct candidate links85 of 150
Requirements lacking direct candidate documentation links65 of 150
Pilot protocols now specified15; none executed

Full versioned review, rationale and gap register (JSON) · Per-link review (CSV) · 15 pilot-test specifications (CSV).

Testing methodology and limitations

Pilot records define an experimental setup, fixture, expected behaviour, metric numerator and denominator, provisional failure criteria, and evidence needed for independent review. Where thresholds are suggested they are exploratory engineering targets, not calibrated international acceptance standards. Sample sizes, statistical power and uncertainty bounds must be determined for each target population and harm severity before any release or certification conclusion.

Unmapped considerations are not presumed irrelevant. Some may require new artefacts, cross-cutting evaluations, social-science designs or retention as open research questions. A missing document does not demonstrate system failure; a candidate document link does not demonstrate system safety.

Priority next stage

Secure independent reviewer assessment of all 165 link decisions; select actual models or controlled test environments; construct licensed, privacy-safe, reproducible evaluation fixtures; register acceptance criteria before running tests; report observed results with counterexamples and uncertainty; and update the documentation register without rewriting previous versions.

Durable evidence preservation and accountability — proposed research requirement

Credible foreseeable-harm warnings, QA findings and consequential decisions should be recorded in a durable, citable, tamper-evident system. Failure to preserve material information can undermine later accountability. Foreseeability alone does not prove deliberate harm. The research problem is to test how well institutions record warnings, attribute decisions, protect evidence and correct errors.

A versioned principle and five candidate tests specifies stable identifiers, provenance, independent backups and recovery drills, integrity checks, warning-to-action traceability, privacy safeguards and correction records. Record durability is not an assertion of indestructibility: access controls, lawful retention, redaction and amendment must remain possible.

Status: proposed design requirement, not a completed test, legal finding or external certification. Record initially registered 13 November 2026.

Versioning and research continuity

This is an early public record of a developing research programme. The companion working dossier, 150-item register, and 15-pilot workbook were prepared as draft files; public hosting of those binaries is a separate step and is not claimed by this page. Future revisions will distinguish proposed, tested, falsified, revised and independently replicated results.

Cross-disciplinary hypothesis: formation, media and conflict-prevention evidence

Potential AI assurance contribution and boundaries

This research extension proposes checks for unsupported claims about moral formation, media exposure, social cohesion, leadership conduct and conflict escalation. AI evaluations should distinguish primary evidence, individual behaviour, causal hypotheses, risk assessment and normative judgment. They must not assign moral worth or infer leaders' mental health from group identity or disputed political acts.

Candidate tests: citation and source-quality review; causal-language calibration; cross-group fairness; misleading narrative detection; warning-to-decision record traceability; independent human verification and corrections.

Falsification: no measurable reduction in unsupported assertions, false warnings or missing provenance compared with existing baseline evaluations.

Shared interdisciplinary framework · Research questions.

Status: proposed extension; no tests executed or independent validation claimed.

Research question

How can AI quality-assurance systems detect and reduce epistemic, institutional and objective failures while preserving human rights, freedom of conscience, autonomy and independently contestable oversight?

Public status: working preprint, not a completed or externally validated paper. Last updated 13 November 2026.