reproducibility-verification

$npx mdskill add yogsoth-ai/de-anthropocentric-research-engine/reproducibility-verification

Verifies experiment reproducibility using ICC and seed-based re-runs

  • Assesses consistency of results across multiple experiment re-runs with varied seeds
  • Relies on statistical tools for ICC calculation and variance decomposition
  • Compares original results to re-run outputs to determine reproducibility thresholds
  • Delivers structured reproducibility assessment with ICC scores and variance breakdown

SKILL.md

.github/skills/reproducibility-verificationView on GitHub ↗
---
name: reproducibility-verification
description: "Verify result reproducibility via re-runs with different seeds and ICC comparison"
version: 1.0.0
category: experiment-execution
type: sop
execution: subagent
prompt: ./prompt.md
used-by:
  - result-analysis
  - result-validation-loop
shared-with: experiment-design
input: experiment design, original results, re-run parameters (n_reruns, seeds)
output: reproducibility assessment with ICC scores and variance decomposition
---

# SOP: Reproducibility Verification

Re-run the experiment with different seeds and compare results to assess reproducibility via ICC and variance analysis.

Subagent — spawned via subagent-spawning/spawn-agent skill.

More from yogsoth-ai/de-anthropocentric-research-engine

SkillDescription
abductive-hypothesis-generationStrategy: 面对异常的最佳解释推理
ablation-brainstormRemove components one by one, observe system changes to reveal hidden dependencies and generate ideas from structural gaps.
ablation-component-mappingMap system architecture to ablatable units for ablation studies
ablation-designDesign ablation studies to isolate component contributions in ML systems
ablation-executionRemove components one by one from a system, record the response/impact of each removal.
abp-vulnerability-classificationClassify assumptions on 2 axes — load-bearing (how much conclusion depends on it) × vulnerable (how likely to be false). Focuses attention on High-Load × High-Vulnerable quadrant.
abstraction-extractionExtract abstract principles from concrete domain cases. Strips domain-specific details to reveal transferable mechanisms.
abstraction-ladderPerform bisociation at multiple abstraction levels
abstraction-ladderingMove between concrete and abstract framings — 3 levels up (Why?) and 3 levels down (How?) to find the most productive research level.
abstraction-to-designAbstract biological principle to design principle. Bridge from biology to engineering.