A headline accuracy value describes one evaluation context. It does not establish that a pathology AI system will behave similarly across laboratories, scanners, preparation protocols, disease prevalence, or patient populations. Bias analysis begins by making those contexts visible and asking which differences could change the system's output or its consequences.
Bias can enter before model training
Sampling decisions, incomplete demographic information, uneven case selection, labeling practice, staining variation, scanner characteristics, and clinical workflow can all shape an evaluation dataset. These factors are not interchangeable. A model may appear stable because the test data resemble the training data, while a shift in site or preparation exposes a different failure pattern.
External validation tests transportability
- Separate development data from evaluation data at the level of patients, specimens, or slides as appropriate.
- Include sites, scanners, preparation protocols, and case mixes that represent the intended setting.
- Report performance by clinically and operationally relevant subgroups rather than only as a pooled value.
- Include uncertainty intervals, missing-data information, and the number of cases behind each estimate.
- Record exclusions and failures so that the evaluation describes the system that was actually tested.
A subgroup result is not automatically a fairness conclusion. Small samples can produce unstable estimates, and a subgroup label may stand in for several correlated factors. Interpretation should connect quantitative results to the intended use, the consequences of error, and the limitations of the available data.
Choose metrics that match the task
Sensitivity, specificity, predictive values, calibration, ranking measures, and agreement measures answer different questions. The relevant choice depends on whether the system classifies a slide, identifies a region, counts cells, supports triage, or produces a quantitative result for review. Thresholds should be described with their context rather than presented as universal quality gates.
External validation is a point in time, not a substitute for lifecycle oversight. Changes in scanners, reagents, protocols, patient mix, labeling practice, or workflow can alter the relationship between inputs and outputs. Monitoring plans should define what is observed, how issues are investigated, who can act, and how limitations are communicated to users.
Fairness work is therefore an evidence and governance practice, not a single score. The most credible evaluation makes the population, setting, data limitations, uncertainty, and known failure modes explicit. It also leaves room to revise conclusions when new external evidence changes what is known.
Sources
- Artificial Intelligence Risk Management Framework AI RMF 1.0 (National Institute of Standards and Technology)
- Good Machine Learning Practice for Medical Device Development: Guiding Principles (U.S. Food and Drug Administration)
- Ethics and governance of artificial intelligence for health (World Health Organization)
Written by
Digital Pathology Solutions Editorial Team
Medical AI and digital pathology
