Our first preserved online research candidate did not pass. GLM-5.3 found real problems, invented some metadata, lacked evidence it should have received, and stopped the article from entering the publication queue. That is a successful gate wrapped around an immature contract.
The run analyzed the same touchdown-regression data as our deterministic launch study. The statistics completed. The charts rendered. Then the specialist reviews produced blocking findings and the workflow withheld the pull request. We did not downgrade the blockers or manually wave the article through.
Rejected means rejected. The candidate article is not part of the publication, even when parts of its analysis may later survive a corrected run.
The biggest failure happened before review
The online researcher had been allowed to generate a new memo from a small registry entry. It replaced the actual expected-touchdown design with a different slope-based study. The deterministic analysis then executed the original design. Reviewers correctly saw a locked memo and an analysis that did not match.
This was an orchestration error, not a prompt-tuning curiosity. A preregistration cannot be a movable target. The corrected pipeline loads the complete question, estimand, temporal design, risks, and required outputs from versioned code before any model call. The researcher role now checks conformance.
The reviewers were also reviewing a partial record
The statistician, adversary, and citation checker had only the memo and analysis summary. They did not receive the provenance record, source URL, license, article, or artifact inventory. The citation checker therefore blocked central numbers as untraceable. It was right under the evidence it had, even though the missing records existed elsewhere in the run.
A review system is only as good as its evidence boundary. We now supply the same complete, explicit bundle to every specialist and identify every file by relative path, byte count, and SHA-256.
The raw artifacts caught a quieter problem
Some role outputs claimed a different model name and a placeholder 2025 timestamp. Those fields came from model-authored JSON, while the run manifest correctly recorded the configured OpenRouter map as z-ai/glm-5.3. We preserved the raw files because the contradiction matters.
The fix is structural: after parsing the schema, transport-side code overwrites the role, exact configured model slug, and current timestamp. The test suite now feeds the client deliberately false self-reported metadata and verifies that none survives.
Six repairs before the next attempt
Lock the question
The preregistration now loads from the versioned series registry. An agent can audit it, but cannot replace it during the run.
Send the evidence
Every specialist now receives provenance, article text, analysis, mechanical checks, and a path/byte/SHA-256 artifact inventory.
Own the metadata
Role, model slug, and timestamp are written by transport-side code after schema validation—not trusted from model prose.
Reconcile every row
A new attrition table accounts for all 950 predictions: 551 evaluated pairs, 257 threshold exits, and 142 pending outcomes.
Name every season
The ambiguous range [2018, 2024] is now the explicit seven-season list from 2018 through 2024.
Use the editor
The editor role now sees the mechanical checks and specialist findings, and any non-pass decision stops publication.
Update: the repaired contract rejected again
The next candidate proved that the first repairs worked. The registration matched, 2025 data arrived with a second provenance record, all 1,078 predictions reconciled, metadata identified GLM-5.3 correctly, and the researcher and citation checker returned detailed passes.
It still did not publish. The statistician requested concrete copy changes around bootstrap interpretation, survivor conditioning, and individual-level advice. More revealingly, the adversarial and editor roles returned literal ... placeholders that satisfied the shape of the JSON schema. Structure was not enough; the contract needed semantic minimums and decision consistency.
We preserved that raw run too. The schema now rejects placeholder-length summaries and findings, a pass cannot hide a blocking finding, and invalid output triggers a retry. The candidate copy also absorbed the statistician's valid revisions before another attempt.
What we did not “fix” by prompt
The reviewers also challenged survivor conditioning, baseline fairness, effect-size usefulness, position scope, and source recency. Those are substantive research questions. The article already bounds several of them, but a stronger v2 should add a rate-based comparator, position splits, attrition by residual quintile, and fresher public data when the source is available.
The contract repair does not guarantee the next run passes. It guarantees the next rejection will be about the study in front of the reviewers, based on the evidence that actually exists, with a record whose identity fields come from code.