The Registered Revisions Pathfinder Pilot was designed to pilot both a new journal policy for peer review and a new framework for randomized evidence generation in science policy. The Registered Revisions policy, a precommitment policy comparable to Registered Reports, enables authors to respond to editor comments implying a change to analysis by providing a Revision Plan for approval before conducting those new analyses. If the plan and other revisions and responses are approved, the manuscript is given an in-principle acceptance. The author enacts the plan, and the manuscript is presumed accepted regardless of the results. This policy aims to focus review on methodological quality while lessening uncertainty, questionable research practices, and publication biases that may occur in late stage peer review.
To evaluate the Registered Revisions policy among an array of journals, we have designed and piloted a “meta trial.” This combines multi-center individually-randomized trial design with prospective meta-analysis. Each journal runs its own randomized trial supported by a centralized backbone design and infrastructure. The results are combined in a planned and predesigned meta analysis.
At the time of this writing, neither the Registered Revisions policy nor the meta trial design have been implemented. This Pathfinder Pilot was designed to develop the materials and iteratively test practical implementations of both to support a scaled implementation.
This document describes the project history through the end of the Pathfinder Pilot, including redesigns, mistakes made, and successes along the way.