Data Analysis

Meticulously collected data can generate worthless evidence if analysis suffers from statistical errors, ignores missing data patterns, or fails to account for confounding variables. Regulators scrutinise analytical approaches because inappropriate methods can obscure safety signals, inflate efficacy claims, and generate misleading conclusions.

Accounting for missing data

Missing data represents one of the most common yet inadequately addressed analytical challenges. Participants drop out, visits are missed, assessments are incomplete, and samples are lost. Ignoring these missing data patterns or handling them inappropriately creates biased results that regulators reject. So, analyse missing data patterns before conducting primary analyses. Understand whether data are missing completely at random, missing at random, or missing not at random. If participants with worse outcomes drop out more frequently than those improving, your completers-only analysis will overestimate treatment benefits. The current standard for handling missing continuous data is multiple imputation. This approach generates multiple plausible datasets with imputed values, analyses each dataset separately, and combines results accounting for imputation uncertainty. Single imputation methods (like last observation carried forward) are generally discouraged because they underestimate variability and introduce bias.

Sensitivity analyses exploring different missing data assumptions demonstrate robustness. If conclusions remain consistent, whether you assume missing data are similar to observed data or systematically worse, regulators gain confidence in the validity of the result. If conclusions change dramatically due to assumptions, this signals fragile findings that will require cautious interpretation. Document dropout reasons thoroughly. Participants withdrawing due to adverse events represent different analytical challenges than those lost to follow-up for administrative reasons. Treatment-related dropouts might indicate a safety concern that requires investigation.

Missing data handling separates rigorous trials from those that merely look rigorous. Ignoring missing data or using outdated methods invites justified regulatory scepticism. Modern missing data methods are accessible and well-documented.

_{Professor Ian White. Statistical Methods for Medicine. University College London}

Hidden confounders

While randomised trials should create balanced groups, chance imbalances can occur, and non-randomised studies almost always have systematic differences between groups. Failing to account for these differences creates confounded comparisons where observed differences might reflect baseline characteristics rather than treatment effects. Assess baseline balance across treatment groups before conducting primary analyses. Examine the demographics, disease severity, other health conditions, and any factors that potentially affect outcomes.
Likelihood score methods help control confounding in non-randomised studies. These approaches model the probability of receiving treatment based on observed covariates, then use likelihood scores to match participants, stratify analyses, or weight observations. Whilst these methods cannot control unmeasured confounders, they substantially reduce bias from measured baseline differences.

Matching statistical methods to data characteristics

Statistical method selection needs to align with data type, distribution characteristics, and study design. Applying inappropriate methods will generate invalid results regardless of data quality. Continuous outcomes require methods appropriate for data distribution. Normally distributed data support parametric approaches like t-tests or linear regression. Skewed data can require transformation or non-parametric alternatives. Time-to-event data requires survival analysis methods accounting for censoring, so consult statisticians during protocol development (not after data collection). Many analytical problems emerge from study design choices that cannot be corrected retrospectively.

Interpreting subgroup analyses conservatively

Subgroup analyses explore whether treatment effects vary across patient characteristics and can be scientifically interesting (but statistically dangerous). Most trials cannot detect subgroup differences, which leads to false positive findings that don't replicate. Pre-specify limited subgroups of genuine clinical interest in protocols.

Test for interaction rather than comparing p-values across subgroups. If treatment effects are significant in one subgroup but not another, this doesn't prove effect heterogeneity. Formal interaction tests assess whether the effect magnitude differs across subgroups. Apply multiple comparison adjustments when examining numerous subgroups. Testing ten subgroups at the 5% significance level creates a 40% probability of at least one false positive finding purely by chance.

Subgroup analyses represent the most common source of overinterpreted trial results. Researchers discover unexpected subgroup differences and construct biological rationales to explain findings that are actually statistical noise. Pre-specification, formal interaction tests, and multiple comparison adjustments are essential safeguards.

_{Professor Doug Altman. Professor of Statistics in Medicine. Oxford University}

Flag underpowered subgroup analyses explicitly. If your trial enrolled 100 subjects and you examine treatment effects in a subgroup of 15, acknowledge that power for detecting meaningful differences is minimal and the results are exploratory.

Unmasking device-specific failure patterns

Medical device trials require device-specific analyses beyond standard efficacy and safety endpoints. Individual device failures, performance variations, or use errors might average out in population analyses while representing important safety concerns. Make sure to analyse device performance at the individual device level. If five of 100 devices malfunction, averaging across all devices obscures this 5% failure rate that might represent an unacceptable risk. Device-level analyses reveal failure patterns, identify problematic production lots, and inform manufacturing improvements.

Examine your learning curves when operator technique affects outcomes. Initial cases might show worse results as operators gain experience, while later cases demonstrate improved performance. Understanding these patterns informs training requirements and shapes regulatory discussions about device usability. Investigate outlier events thoroughly. Extreme values might represent measurement errors that require correction, or genuine device or patient events that need investigation (never discard outliers without documented justification).

Pre-specifying analytical approaches

Statistical analysis plans documenting analytical approaches before viewing outcome data prevent data-driven analysis choices that inflate false positive rates. These plans specify primary and secondary endpoints, statistical methods, missing data handling, subgroup analyses, and sensitivity analyses. Develop detailed analysis plans during protocol development and finalise them before database lock. Document all analytical decisions, including handling of protocol deviations, missing data, outliers, and any departures from planned analyses.

Transparent reporting builds credibility.

The data analysis workshop

Data analysis determines whether trials generate evidence that regulators accept and clinicians trust. Our methods will help you to account for missing data using appropriate modern methods. Adjust for baseline differences, creating unbalanced comparisons. We can help you to match statistical methods to data characteristics, interpret subgroup analyses conservatively, and unmask device-specific failure patterns.

Waypoint checklist

Data analysis for a MedTech device should consider:

• Missing data traps that account for dropouts/lost samples
• Hidden confounders that you should adjust for baseline differences
• Match methods to the data type, or your stats will prove wrong
• Overinterpreted subgroups, so flag any underpowered claims
• Unmask any device-specific failures
• Poor analysis risks wasted trial or FDA pushback