SaMD Population Performance Red-Team.
An independent stress-test of your Software-as-a-Medical-Device algorithm against the populations your training data under-represents, the exact evidence MHRA and notified bodies now expect.
A population-fit performance dossier, subgroup performance breakdown, and prioritised remediation recommendations for your algorithm.
SaMD manufacturers and AI medical-device developers preparing for MHRA submission, NICE ESF assessment, or defending a live UK approval.
Why this exists
The 2024 UK Equity in Medical Devices Independent Review, chaired by Dame Margaret Whitehead, confirmed pulse oximeters may be less accurate in patients with darker skin, a finding replicated in a 2026 pulse oximeter study. Notified bodies, HTA reviewers and NHS procurement now read every SaMD file through this lens.
The MHRA Inclusion and Diversity Plan and NICE Evidence Standards Framework Standard 4 both now require developers to demonstrate subgroup performance and disclose dataset diversity. Files that cannot answer the subgroup question are delayed, deferred, or declined.
The Red-Team is not a light-touch audit. It is an adversarial evaluation designed to find where your algorithm underperforms, before MHRA, NICE, a notified body or NHS procurement does.
What is included
- Population-fit performance dossier, mapping your algorithm's real-world performance against each of the eight social-factor subgroups
- Subgroup performance breakdown, including confidence intervals, at the level notified-body and NICE ESF reviewers ask for
- Adversarial testing protocol: how we constructed the test set, how we selected corner cases, how we validated results
- Remediation recommendations, prioritised by clinical risk
- Independent-review letter, signed by an ex-notified-body reviewer on our team
- Handover session with your ML/product team to walk through findings
What is not included
- Retraining or fine-tuning your model, that is your ML team's scope
- Access to our raw participant data, we can license this separately under a dataset access agreement
How it works
Weeks 1 to 2. Intake. You share your intended use, training-data provenance, existing performance dossier and target regulator. We map the subgroups where under-representation is highest-risk.
Weeks 3 to 8. Adversarial test-set construction, drawing on the Equity Engine panel. Independent evaluation. Corner-case validation.
Weeks 9 to 12. Dossier drafting, prioritisation of remediation, sign-off, independent-review letter, handover.
Where this fits
Priced against the cost of a single mid-size regulatory-affairs consulting engagement, and roughly one-third of what an average SaMD hold from a notified body costs to remediate. Fixed, so your regulatory-affairs, ML and commercial teams can budget it from a single line item.
Take the Regulatory Readiness Scorecard.
Twenty questions, six domains. A plain-English diagnosis of your regulatory exposure, and the specific fixed-price engagement that closes each gap.