# Confirmatory preregistration readiness V2 — draft, not registered

Prepared: 2026-10-07T04:07:12.795830-07:00
Status updated: 21 October 2026 (America/Los_Angeles).

This preserves review item 17 and separates development apparatus from confirmatory research. No target outcomes, task bodies, keys, pricing approvals or model authority are included. Existing original H1 collection limits remain unchanged; this document grants no dispatch authority.

The primary question is the incremental decision value of registered cross-verifier interaction features beyond matched marginal information under fixed total cost and one prospectively specified task-family shift. Agreement alone, nominal verifier count, covariance summaries and candidate-score calibration are insufficient evidence for selection utility.

## Required before an immutable registration

- Identify exact source and held-out task families, licenses, exclusions, contamination/exposure checks and authenticated scoring suite revisions. Target outcomes must remain inaccessible to fitting and policy selection.
- Qualify exact generator/judge model, executable, weights, template/tokenizer, rendering, original options and usage measurement; pin manifests. Complete real development collection before source-only fitting.
- Freeze estimators, additive/joint interaction features, fit/calibration splits, stopping rules, action inventories and both single-judge baselines. Include fixed voting, matched-retention/robust published selection and execution controls where applicable; record faithful-baseline applicability or exclusions prospectively.
- Primary shared-bank replay: identical ordered candidate bank and outcome suite; policies reveal only paid candidates/judgments. A separate pure-selection diagnostic holds revealed candidates and generation cost common. Adaptive generation is a separate future experiment, not inferred from replay.
- Freeze total-cost definition, primary budget and secondary grid using measured development usage and conservatively bounded reservations. Charge generation, verification, repeated context, retries/failures, execution and source-only fitting separately. Unknown actual cost cannot become zero. Reconcile all CLI spending before any additional paid launch; overall external spending cannot exceed $100.
- Specify task count and within-task seeds using a prospective development-based power calculation. A three-percentage-point minimum effect is a planning target, not an expected gain. A feasibility failure or insufficient headroom may stop progression without replacing the confirmatory question post hoc.
- Primary endpoint: paired task-level hidden-test success, abstention counted as failure. Specify exact seed aggregation, task-cluster interval method, primary test and multiplicity treatment for secondary comparisons. Archive analysis source and authored validation fixtures before target results.
- Specify completion, timeout, infrastructure failure, exclusion and stopping rules, including treatment of missing candidate/judge/hidden-test outcomes. Preserve failed requests and unknown charges. Do not silently condition analysis on completed successful runs.
- Freeze all exploratory analyses separately: candidate coverage/oracle ceiling, failure atlas, joint false acceptance/rejection strata, risk/coverage and dependence diagnostics. Hidden truth and oracle information remain offline; no target fitting, stopping or generation feedback.
- Archive timestamped immutable protocol, exact source/config hashes and analysis plan in a durable publicly accessible registration. A local draft, Git commit, self-published proposal or AI-assisted review is not completed preregistration. Report amendments and deviations without retroactive relabeling.

## Latest serving identity preflight — 21 October 2026 Pacific

The latest operator-provided Warden preflight was reported on 21 October 2026 Pacific; the shell output did not preserve its run time. The named Ollama process executable identity matched. Loaded-library qualification and tokenizer parity remained false, candidate calls remained disabled, and zero model calls occurred. This verifies process executable identity only; it does not establish loaded model weights, complete native-library closure, tokenizer/request-template parity, or any verifier-study result. The frozen 24-task MBPP/BigCodeBench H1 collection remains unrun.

## Current unfulfilled gates

Loaded-library qualification and served tokenizer/request-template parity remain open; the latest Warden preflight matched the named process executable only. Genuine pricing/settlement and reconciled total spend; full owned runtime/capture/cleanup qualification; real development collection and fitted policies; empirical candidate headroom and task-family coverage; faithful strong-baseline implementation; powered affordable confirmatory budget; and immutable external registration also remain open. No confirmatory run is authorized by this draft.

Negative, inconclusive and feasibility-stop outcomes remain valid research outcomes. This readiness checklist does not assert novelty, frontier-model transfer or independent empirical validation.

## Required distinctions from the original diagnostic collection

Original diagnostic H1 remains 24 tasks (6 development, 6 calibration, 12 target-diagnostic), eight candidates per task, three generation/judge roles and 576 registered call slots, with its original 14,400-second inclusive wall clock, 7,680-second aggregate grading bound and all original case/native/cost/stop/no-resume limits. A new joint-verifier pilot or powered confirmatory study needs its own prospectively priced, source/task/config-bound authorization. Historical 120-task engineering-pilot outputs and the original target-diagnostic split cannot silently become new confirmatory evidence or enlarge the original collection grant.

## Actual computational object to register

The currently prepared policy fits source-only ridge regression of continuation-dependent incremental action value: correctness after an action and fixed continuation minus incumbent correctness before that action. It is not a calibrated joint correctness probability or a general portfolio VOI estimator. The additive arm has 31 registered features; the joint arm adds (score_A - 0.5) * (score_B - 0.5), raw zero if either score is missing, then source-history standardization. Source tasks have equal total fitting weight; standardization uses unweighted history rows. Pin exact feature definitions, source revision, action/continuation structure, regularization, preprocessing and missingness conventions. Match inputs, estimator class, training/calibration, actions and stopping across the ablation.

The ordering is development-only fit, separate source calibration, coefficient/threshold lock, immutable registration, then untouched confirmatory target evaluation. Once both scores are observed, neither judge can be queried again for that active candidate; the feature concerns further generation, public testing or stopping. A positive interaction-feature result would not alone isolate error correlation from nonlinear calibration, difficulty or other interactions. General verifier-portfolio allocation requires a distinct prospective protocol.

## Fixed missing-outcome semantics

Intentional abstention counts as endpoint failure. Missing or ambiguous execution outcomes remain unknown, not false and not discarded as complete-case exclusions. Report paired-gain, candidate-coverage and headroom partial-identification bounds using shared candidate constraints. Hidden-outcome oracle bounds are offline diagnostics, never an imputed oracle or controller input. Additional uncertainty and sampling inference require separately specified task-level procedures; authored small-bank bounds do not certify real outcomes.

V1 remains immutable. This successor addresses review against the current personal proposal; it remains an unregistered readiness document and does not authorize execution.

Source prepared with AI assistance; separate model-assisted review verifies scoped consistency, not outside peer review or empirical validation. Source SHA-256: 37da0f8763a942872bfd6b2652ae9041de1ea346ad3af495713498f4f371c1c2. Review receipt SHA-256: 4d7a94de9027586d65bd0d2d82f9f31d16a1befba2e94c6378ab55b0c9e4250f.