in silico
Phase 1: In Silico Validation on Public & Simulated Data
- Objective
- To rapidly falsify the core assumption that SuStaIn can recover distinct progression subtypes from longitudinal PFT data, using publicly available datasets and synthetic data with known ground truth.
- Estimated cost
- €500-2000
- Estimated duration
- 4-6 weeks
- Success criteria
- ARI on synthetic data (K=2, N=500, 3 time points) · ≥ 0.80 · (Adjusted Rand Index between true subtype labels and pySuStaIn MAP assignments)
- Bootstrap ARI on COPDGene (K=2) · ≥ 0.70 (mean across 50 bootstraps) · (ARI between original subtype assignments and assignments from each bootstrap resample)
- Minimum sample size for ARI >0.70 · N ≤ 500 · (From synthetic sensitivity analysis: find smallest N where mean ARI >0.70)
- Go if
- All three success criteria met (synthetic ARI ≥0.80, bootstrap ARI ≥0.70, required N ≤500)
- No-go if
- Synthetic ARI <0.60 OR bootstrap ARI <0.50 OR required N >1000
- Pivot if
- ARI acceptable but only with >4 time points or >5 years follow-up: redesign Phase 2 to use richer datasets (e.g., SPIROMICS with 5 visits)
- Risks
- COPDGene/SARP data access delayed >4 weeksProbability: mediumMitigation: Use synthetic data only for initial validation; also download publicly available ECLIPSE (NCT00292552) summary statistics if full data unavailable
- pySuStaIn fails to converge on real PFT data (e.g., due to non-monotonic biomarkers)Probability: mediumMitigation: Pre-screen biomarkers for monotonicity using linear mixed models; exclude biomarkers with >20% non-monotonic trajectories
- Synthetic data too simplistic (e.g., Gaussian noise underestimates real variability)Probability: lowMitigation: Add heteroscedastic noise proportional to FEV1 level, and simulate missing data patterns from real COPDGene dropout rates