O0-RP-006

Research Methodology: Simulation-First Pipeline

**Version:** 1.0

**Research status:** APPROVED

CLAIM STATUS: METHODOLOGICAL
EVIDENCE TYPE: GOVERNANCE STANDARD
PHYSICAL VALIDATION: NONE
INDEPENDENT REPLICATION: N/A
PHILOSOPHICAL PROVENANCE: O/0 ARCHIVE
ARCHIVE ENDORSEMENT: LIMITED TO REPORTED RESULT

Abstract

This document specifies the complete research methodology for the O/0 Research Program, defining a simulation-first pipeline that transforms philosophical claims into empirical computational results. The pipeline consists of seven mandatory stages: claim extraction, formalization, null model construction, simulation execution, statistical analysis, adversarial review, and reporting. Each stage has explicit entry criteria, exit criteria, quality gates, and decision rules that determine whether a research question advances to the next stage or is returned for revision. The methodology ensures that every published result is reproducible, statistically rigorous, and epistemically bounded.

Source proposition

The O/0 archive generates claims that are expressed in natural language and phenomenological terminology. The research program requires a systematic method for converting these claims into formal, testable propositions without introducing artifacts from the translation process.

Scientific audit

Simulation-based science faces well-documented methodological challenges: overfitting to model assumptions, confirmation bias in parameter selection, lack of null-model discipline, and irreproducibility due to undocumented computational environments. This pipeline incorporates best practices from computational physics, artificial life research, and pre-registration methodology to mitigate these risks.

Research question

What is the minimal complete pipeline that converts a philosophical claim into a rigorously bounded computational result while maintaining full reproducibility and epistemic honesty?

Operational definitions

1. **Pipeline stage**: A discrete phase of research with defined inputs, outputs, entry criteria, and exit criteria.

2. **Quality gate**: A mandatory checkpoint that must be passed before advancement to the next stage.

3. **Decision rule**: A pre-specified criterion that determines the outcome of a quality gate (PASS, REVISE, or TERMINATE).

4. **Null model**: A minimal model that reproduces the statistical properties of the observed phenomenon without invoking the proposed mechanism.

5. **Effect size**: A quantitative measure of the magnitude of a result, independent of sample size.

6. **Pre-registration**: The act of formally specifying hypotheses, methods, and analysis plans before data collection begins.

Hypothesis

Not applicable. This is a methodological governance document.

Null hypothesis

Not applicable.

Competing explanations

Alternative methodologies (e.g., theory-first, experiment-first, data-driven discovery) exist and may be appropriate for other research programs. The simulation-first approach is chosen because the O/0 program's claims are not yet amenable to physical experimentation but can be translated into computational models. This is a pragmatic choice, not a claim about the superiority of simulation over experiment.

Formal model

Stage 1: Claim Extraction

**Input:** Source passage from the O/0 philosophical archive.

**Process:**

1. Identify the specific propositional content (strip metaphor, poetry, and rhetorical framing).

2. Decompose compound claims into atomic propositions.

3. Classify each proposition using the taxonomy from RP-003 (established fact, philosophical interpretation, mathematical analogy, speculative hypothesis, untestable assertion).

4. Select only propositions classified as "testable hypothesis" or "mathematical analogy (tractable)" for pipeline entry.

**Output:** A numbered list of atomic, classified propositions with tractability assessments.

**Quality Gate 1:** At least one proposition must be classified as tractable (tractability ≥ MODERATE). If no tractable propositions exist, the claim is returned to the archive with a note explaining why it cannot currently be investigated.

**Decision Rule:** If PASS → Stage 2. If no tractable claims → TERMINATE with documentation.

Stage 2: Formalization

**Input:** Tractable atomic propositions from Stage 1.

**Process:**

1. Express each proposition as a formal mathematical or computational statement.

2. Define all variables, parameters, and their domains.

3. Specify the formal hypothesis H₁ and the null hypothesis H₀.

4. Identify all assumptions required for the formalization and document them explicitly.

5. Verify that the formalization is faithful to the source proposition (no inadvertent weakening or strengthening).

**Output:** A formal specification document containing H₁, H₀, all variables, parameters, domains, and documented assumptions.

**Quality Gate 2:** The formalization must be reviewed by at least one team member who did not perform the formalization, confirming that (a) the formal statement is a faithful translation and (b) the formal statement is in principle falsifiable.

**Decision Rule:** If PASS → Stage 3. If unfaithful → REVISE. If unfalsifiable → TERMINATE.

Stage 3: Null Model Construction

**Input:** Formal specification from Stage 2.

**Process:**

1. Construct a null model M₀ that explicitly lacks the proposed mechanism but matches all other properties of the test model.

2. Verify that M₀ is the simplest model that could produce the observed outcome by alternative means.

3. Identify at least two additional competing models M_alt that could explain the predicted result via different mechanisms.

4. Specify the statistical test that will discriminate between H₁ and H₀.

5. Compute the required sample size (number of simulation runs) for the target effect size and power (β ≥ 0.8, α = 0.01).

**Output:** Null model specification, competing model specifications, statistical test specification, power analysis.

**Quality Gate 3:** The null model must be capable of producing the predicted result (otherwise the test is trivial). This is verified by running a small pilot of M₀ to confirm it has non-zero probability of producing patterns that could be mistaken for the target phenomenon.

**Decision Rule:** If PASS → Stage 4. If null model is degenerate → REVISE. If no discriminating test exists → TERMINATE.

Stage 4: Simulation Execution

**Input:** Formal models (test, null, competing), statistical test specification, sample size requirements.

**Process:**

1. Implement all models in the program's standard computational framework.

2. Record all random seeds, library versions, hardware specifications, and execution parameters.

3. Run the pre-specified number of replicates for each model condition.

4. Store all raw output data in the program's archival format.

5. Perform no analysis during execution (prevent data peeking).

**Output:** Raw simulation data, execution logs, environment specification files.

**Quality Gate 4:** All runs must complete without error. Any crashed runs are documented and excluded with justification. The total number of successful runs must meet the pre-specified sample size.

**Decision Rule:** If sufficient runs complete → Stage 5. If <80% of planned runs complete → INVESTIGATE and REVISE implementation.

Stage 5: Statistical Analysis

**Input:** Raw simulation data from Stage 4.

**Process:**

1. Apply the pre-specified statistical test to the data.

2. Report effect sizes with confidence intervals, not just p-values.

3. Perform sensitivity analyses (vary thresholds, parameters, analysis methods).

4. Apply multiple comparison corrections if more than one test is conducted.

5. Compute Bayes factors in addition to frequentist statistics.

6. Document any deviations from the pre-registered analysis plan and justify them.

**Output:** Statistical results report with effect sizes, confidence intervals, p-values, Bayes factors, and sensitivity analyses.

**Quality Gate 5:** Results must pass a statistical review confirming correct application of tests, appropriate corrections, and honest reporting of all analyses (including null results).

**Decision Rule:** If statistically sound → Stage 6. If errors found → REVISE analysis. If data integrity issues → return to Stage 4.

Stage 6: Adversarial Review

**Input:** Statistical results from Stage 5.

**Process:**

1. Apply the Adversarial Standards Protocol (RP-007) to the results.

2. Generate at least three alternative explanations for the observed results.

3. Identify the weakest assumption in the inference chain.

4. Attempt to reproduce the result using the null model with different parameters.

5. Check all results against the Prohibited Claims Register (RP-004).

6. Document all adversarial challenges and responses.

**Output:** Adversarial review report documenting challenges, responses, and remaining vulnerabilities.

**Quality Gate 6:** The adversarial review must be conducted by a reviewer who was not involved in Stages 1-5. At least one genuine vulnerability must be identified and addressed (to prevent rubber-stamping).

**Decision Rule:** If adversarial challenges are addressed → Stage 7. If fatal vulnerability found → return to appropriate earlier stage. If prohibited claim detected → TERMINATE or REVISE.

Stage 7: Reporting

**Input:** All outputs from Stages 1-6.

**Process:**

1. Compile the complete research record following the Record Schema (RP-009).

2. Include the claim-status banner with honest epistemic assessment.

3. File all raw data, code, and analysis scripts in the program repository.

4. Cross-reference with the Claim Inventory (RP-003) to update tractability assessments.

5. Submit for final editorial review.

**Output:** A complete, publishable research record (O0-SIM-XXX or O0-MATH-XXX).

**Quality Gate 7:** The final document must pass automated schema validation and human editorial review.

**Decision Rule:** If valid and complete → PUBLISH. If incomplete → REVISE. If schema violations → REVISE.

Controls

The pipeline itself is controlled by tracking the rate at which claims successfully traverse all seven stages. If the pass-through rate exceeds 90%, the quality gates are likely too permissive. If it falls below 10%, the gates may be too restrictive or the source claims may lack tractability.

Predictions

The pipeline predicts that approximately 20-40% of philosophical claims that enter Stage 1 will successfully produce a published research record. The majority of terminations are expected at Stage 1 (untractable claims) and Stage 3 (no discriminating test available).

Falsification criteria

The pipeline methodology is inadequate if: (a) it produces published results that are subsequently shown to be artifacts of the methodology itself, (b) it systematically fails to detect known errors when they are deliberately introduced (tested annually via red-team exercises), or (c) it proves so cumbersome that no research is produced within a reasonable timeframe.

Results / Expected Outcomes

The seven-stage pipeline has been fully specified with entry criteria, exit criteria, quality gates, and decision rules for each stage. The pipeline is immediately applicable to all new research entering the program.

Uncertainty

The appropriate strictness of quality gates is not known a priori and will require calibration based on the first several research cycles. Decision thresholds may need adjustment.

Limitations

The pipeline is designed for computational simulation studies. Extensions to mathematical proof-based studies (O0-MATH series) require modified quality gates at Stages 4 and 5. Extensions to observational studies (O0-OBS series) require additional ethical review stages not specified here.

Replication status

Governance document. The pipeline has been retrospectively applied to O0-SIM-001 through O0-SIM-005 and found to be consistently applicable, though some early records would not have passed all quality gates under the current specification.

Data and code

A pipeline-tracking spreadsheet template and automated schema-validation script are maintained in the program's code repository.

Relationship to philosophical archive

The pipeline's first stage explicitly engages with the philosophical archive as source material. However, the archive's authority ends at Stage 1: once a claim is extracted and formalized, its subsequent fate is determined entirely by empirical and logical criteria. Conceptual provenance from the archive does not constitute empirical support for any claim.

References

• Munafò, M.R. et al. (2017). A manifesto for reproducible science. Nature Human Behaviour.
• Nosek, B.A. et al. (2015). Promoting an open research culture. Science.
• Grimm, V. et al. (2006). A standard protocol for describing individual-based and agent-based models (ODD protocol).
• Gelman, A. & Loken, E. (2014). The statistical crisis in science. American Scientist.
• Simmons, J.P., Nelson, L.D., & Simonsohn, U. (2011). False-positive psychology.
• Bedau, M.A. (2003). Artificial life: Organization, adaptation, and complexity from the bottom up.

Revision history

• v1.0: Initial document generated.