RESEARCH TRANSFER · PHASE 6 · PROPOSED VALIDATION PROTOCOL · 5 NOVEMBER 2026
From Recorded Findings to Comparative Evidence
A successor-ready experiment specification: what the next researcher should test, why, with which safeguards and what outcomes would justify continuation, correction, rejection or bounded completion.
Research transfer & perimeters · Original milestones · Original decisions
Evidence status and novel question
Proposed work, not executed results. As of the existing 5 November record, v1.1 has internal reproducibility evidence and zero completed independent v0.9 reviews. Three candidate approaches—frozen baseline, hybrid extraction and evidence-first—have not been comparatively implemented and evaluated here. This is a prospective protocol, not proof that either alternative improves accuracy or safety.
Question: Can a system improve interpretation of unfamiliar natural-language claims, including temporal, scope and evidentiary variation, while preserving typed-reasoning correctness, calibrated abstention, attributable sources and human oversight, at an acceptable quality-adjusted cost?
Frontier-level synthesis hypothesis, unvalidated: Evaluation should measure not only answers but the discoverability of their failure modes. When an answer changes, can a successor determine whether the cause was source revision, extraction error, schema mismatch, model replacement, missing evidence or a changed legal requirement? Record the causal trace necessary to discriminate these explanations; do not treat the hypothesis as a documented scientific discovery.
Three architectural candidates and different objectives
No design is automatically superior: the preferred choice depends on semantic accuracy, rights, safety, cost, intended domain, auditability and how uncertainty is handled. A research-only system and a high-stakes production service need different release criteria.
Falsifiable hypotheses and counter-evidence
H1 · Extraction bottleneck
A revised extractor improves field-level agreement and relation coverage on previously unseen inputs versus A, without increasing unsupported confident decisions.
Refutation: gains disappear on blind independent cases, or false confident assertions increase materially.
H2 · Provenance-first control
Explicit source identity, observation time, validity period and correction history reduce falsely classified contradictions and unsupported conclusions versus A.
Refutation: no source-grounding gain under fair conditions, or new false confidence from misleading provenance.
H3 · Observability and transfer
Field-level error taxonomy and versioned decision logs reduce the time and ambiguity needed to diagnose failures when a successor reviews the system.
Refutation: reviewer tracing is no faster, more accurate or more consistent than a clearly specified simpler log.
H4 · Evaluation coverage
A stratified, independently sourced dataset reveals failures concealed by the existing developer-authored synthetic examples.
Refutation/limit: if no differences are observed, report the study's power and coverage limits; do not infer complete generality.
Proposed experimental order, without restarting previous work
1. T1: Frozen baseline reproduction
Run existing documented local checker in authorised environment; record command, dependency versions, original hashes and output
Evidence gate: Reproduces baseline or raises a discrepancy; do not silently patch original evidence
2. T2: Review-pilot adjudication
Have two independent reviewers label the existing 12 *exposed fictional* cases with separate original files
Evidence gate: Reports pilot agreement and ambiguous cases, not new-source semantic generalisation
3. T3: New blind case set
Independently construct and document a rights-cleared, non-exposed and stratified evaluation sample
Evidence gate: Frozen inputs and reference annotations before comparison; conflicts disclosed
4. T4: Candidate isolation
Hold reasoning contract constant while swapping extraction for B; separately test provenance module for C
Evidence gate: Can attribute changes to tested component, subject to interaction effects
5. T5: Error and safety analysis
Evaluate abstention, false contradiction, unsupported entailment, provenance, sensitive-data treatment and human-review events
Evidence gate: Per-domain risk report; no aggregate hides unacceptable high-impact errors
6. T6: Quality-resource Pareto review
Compare performance alongside latency, cost, manual annotation and audit overhead
Evidence gate: Claims of efficiency only at comparable evidence quality and uncertainty
7. T7: Reproducibility/transfer
Give an independent successor frozen files, manifests, methods, schema and a step-by-step handover
Evidence gate: Successor can repeat or clearly state missing dependencies and barriers
The existing exposed 12-case packet is a reviewer-process pilot. An unseen corpus with independently established reference labels is a separate prerequisite for stronger generalisation claims. Do not manufacture missing historical labels, rerun an outdated workflow merely to increase pass counts, or promote a draft to a validated result.
Measurement and mandatory reporting boundaries
Decision rule to be fixed before data inspection: Define minimum acceptable changes for task-specific primary outcomes and non-inferiority for high-severity error rates; set stopping rules for privacy, unsafe autonomous action and provenance falsification. This page deliberately does not invent numeric thresholds without a validated risk setting and evaluation sample.
Safety and regulation are implementation-dependent
Offline synthetic research
Focus on data integrity, reproducibility, annotation independence, representational limitations and truthful reporting. Do not claim live-system compliance.
Third-party APIs or hosted models
Assess source confidentiality, provider changes, logging, subprocessors, permissions and supply-chain accountability.
Autonomous multi-stage agents
Assess scoped tool identity, delegation, cumulative errors, human approvals, traceability, interruption and recovery.
Public or regulated decisions
Identify actual deployment, sector duties, legal effective dates, privacy safeguards, equal-treatment risk and contestability. A generic research test is not a conformity assessment.
Current legal obligations are jurisdiction- and date-specific. The EU AI Act follows a progressive timetable: Article 50 transparency duties apply from August 2026, while relevant high-risk regimes have later dates. See the official AI Act implementation timeline. The NIST AI RMF GOVERN 1.6 inventory guidance informs system discoverability but is not itself a universal statutory mandate.
What “360°” can responsibly mean
An exhaustive historical/current/future account cannot be honestly declared. Instead, describe the coverage frame and its omissions: time (historical, current, horizon scan), domain, industry, country, modality, model architecture, stakeholder, research lifecycle, evaluation dimension and evidentiary status. Each new sample or claim should have an explicit inclusion/exclusion rationale, source date, rights status, verification state and unresolved alternatives.
Suggested new research object: a coverage exception record. For every unexamined domain or inaccessible source, retain the reason, risk importance, what would be needed to investigate it, and whether its absence could change the current conclusion. This makes incompleteness inspectable instead of falsely hiding it beneath broad site coverage.
For agent-duration research, METR's task-completion time horizons provide a bounded external comparison; its metrics concern a task suite and do not measure universal AI capability or every form of autonomous activity.
Transfer package: what a successor must be able to distinguish
- What happened: documented observations and source records, not inferred successes.
- What was learned: falsifiable implications, separated from generalisable claims.
- What changed: architecture choices and reasons for accepting or rejecting alternatives.
- What remains unproven: absent reviewer annotations, generalisation, risk and comparative gates.
- What can be reused: frozen files, tests, protocols, error categories, stakeholder context and links.
- What can be revised: candidate extraction or provenance architecture without rewriting historical artefacts.
- What ends the phase: an accurate outcome record and limitation statement, whether positive, negative or inconclusive.
Suggested closure states: completed and validated; completed with bounded evidence; methodologically ready but blocked by independent evidence; discontinued with reason; transferred for future investigation. A failed experiment with documented sources, methods, limitations and design implications may be a complete research contribution.
Related work and original evidence
Published 5 November 2026 · Prospective experiment specification and successor decision framework. No new blind results, independent reviews, software experiments, commercial deployment or scientific novelty certification were conducted to produce this page.