The gate said yes. Visual QA said no.

Why model approval was necessary but not sufficient—and why we withheld an otherwise passing run.

Our first GLM-5.3 candidate failed because the contract was weak. The second failed because the semantic gate was weak. The repaired run passed every specialist and every mechanical check. Then visual QA found stale numbers inside two chart titles, so we withheld it.

The chart bars were computed from the current 2025 analysis, but their visible headlines still said 8.9% and 2.8 touchdowns—the prior launch values—instead of 11.7% and 3.0. The PNG hashes were valid. The claims visible to a reader were not.

THE RELEASE RULE

A passing research gate makes a run eligible for publication. It does not prove the website is presenting that run consistently.

What passed

The registered study used regular-season WR/TE data from 2014 through 2025 and an expanding-window test for each season from 2018 through 2025. It produced 1,078 walk-forward predictions. Of those, 649 became evaluable consecutive-season pairs, 301 fell below the next-season 40-target threshold, and 128 await 2026 outcomes.

The opportunity baseline reduced mean absolute error from 2.65 touchdowns to 2.34, an 11.7% improvement. The paired difference was 0.31 touchdowns with a 95% bootstrap interval from 0.18 to 0.45. The reviewed article says what that is: a modest population-level benchmark improvement. It is not a complete projection, a causal estimate, or a licensed excuse to fade an individual player mechanically.

What the five roles actually did

The researcher verified the locked registration and row reconciliation. The statistician checked temporal order, paired outcomes, attrition, uncertainty, and baseline fairness. The adversarial reviewer tried to overturn the headline and highlighted over-application risk. The citation checker matched central claims, figures, source URLs, hashes, and license language. The editor received all four reviews plus the mechanical checks and found no concrete prepublication change outstanding.

All five roles used z-ai/glm-5.3 through OpenRouter. That is role separation, not reviewer independence. Shared-model blind spots remain a limitation, so we call these specialist-agent reviews—not human peer review.

The bug was outside the model desk

The reviewers received figure paths and cryptographic hashes, not a semantic description of the words rendered inside each image. They could verify that a file existed and had not changed; they could not compare the pixels' headline to analysis.json. That is a contract failure, not a statistical or prompting failure.

We removed analyzer-side copying into public/charts. Figures now live under the immutable run ID. More importantly, every new PNG is generated from figure-specs.json, which records its exact title, source metrics, and final image hash. Both the research gate and the promotion command recompute those relationships independently.

The five-stage publication contract

01

Compute

Deterministic Python acquired and hashed two nflverse assets, executed the registered walk-forward model, reconciled attrition, and created the article candidate and figures.

02

Review

Five schema-constrained GLM-5.3 roles audited registration conformance, statistics, adversarial risks, citations, and editorial scope.

03

Gate

Code required every mechanical check, every role decision, and zero blocking findings. Models could not waive a failed condition.

04

Promote

A separate command requires a figure contract, re-hashes the complete bundle, and updates article metadata, the run ledger, and version-addressed figures together.

05

Deploy

Only the merged promotion state can enter the Render build, where the evidence synchronizer exposes the exact reviewed bundle.

One missing receipt stays missing

The accepted run happened before we added openrouter-call-ledger.json. Its manifest records the configured model for every role and each final review records model identity, but the bundle does not contain exact prompt tokens, completion tokens, latency, router response IDs, or billed cost. We cannot reconstruct those values reliably, so we do not estimate them and call the estimate a receipt.

Future calls flush a public-safe ledger after every attempt. It records returned usage and cost, timing, model IDs, and SHA-256 receipts for the prompt, evidence, schema, and raw response. It excludes keys and payload text. Rejected retries count toward the bill instead of disappearing behind a successful final answer.

Inspect the release

AI-005

The method is part of the product.

All process notes →