Artificial Intelligence and Machine Learning for In Vitro Diagnostics (AI/ML) is revolutionizing medical device development and is being leveraged in applications ranging from digital image analysis to in vitro diagnostics. Although it is expected that the number of AI/ML-enabled medical device products will increase exponentially in the future, there is currently no suitable regulatory framework for addressing the iterative improvements that can occur post-launch for AI/ML-enabled medical devices, including in vitro diagnostics, while ensuring their safety and effectiveness for their intended use. Such a regulatory framework would need to take into account numerous factors, such as the potential use of Real World Data (RWD), and the appropriate metrics to use to address the unique challenges posed by AI/ML applications used as software in a medical device (SiMD) and/or software as a medical device (SaMD).
Artificial Intelligence and Machine Learning (AIML) for IVDs Working Group:
This Working Group is scoping a new project for the application of Artificial Intelligence and Machine Learning for in vitro diagnostics.
Deliverables
-
Glossary
The Work Group will deliver a glossary of terms associated with AI/ML. The glossary will be developed by assessing currently available terms and definitions found in documents published by national and international agencies as well as standards-setting bodies. Whenever possible, currently available defined terms will be used, and where inconsistencies in terms or definitions are found, they will be noted. The glossary will provide a standardized vocabulary that will be utilized throughout the other documents developed by the Work Group. In addition, a publication repository will be created as a resource.
-
Software pre-specifications (SPS) template
The Work Group will create an SPS protocol template that will enable a software developer to explain the anticipated modifications the developer plans to implement once the software is in use. The protocol should be flexible to enable the incorporation of a variety of changes that developers may implement and needs to address the “region of potential changes” around the initial specifications and labeling of the original software. Elements of such a protocol may include: a description of anticipated change and its justification; a description of the impact of the change to the software’s intended use (if any); and a comparison table that contrasts the software before and after the anticipated software change(s) and required mitigation controls. Consideration will also be given to how such a template would differ for SaMD versus SiMD, and to requirements for a QMS that will enable a total product life cycle (TPLC) approach like that shown below from the FDA’s discussion paper.
-
Algorithm change protocol (ACP) template
The Work Group will create an ACP template that will include, for example, a monitoring plan; data management plan; algorithm training plan; algorithm update plan; and a rollback plan.
As shown in the table below from the FDA’s AI/ML discussion paper, example components of each of the plans listed above include:
● Monitoring plan: Definition of appropriate monitoring timeframes, triggering parameters, and acceptable performance criteria
● Data management plan: Procedures and systems for separating training and validation data and procedures for storing training and validation performance records for all models
● Algorithm training plan: Procedure for training using the same inputs; new input discovery and updates; and training on new populations
● Algorithm update plan: Performance criteria for proceeding to update; update procedures; and communication plan
● Rollback plan: Rollback criteria and backup and recovery procedures
-
Framework
The Work Group will deliver a framework document that will include the Glossary, SPS template, and ACP template, as well as additional background and context.
Who is working on this project?
MDIC has assembled a work group comprised of member organizations and other subject matter experts to guide work on this project.
Industry:
Hamed Amini, GRAIL
Pat Baird, Philips
Jeff Ballyns, PhD, Becton Dickinson
Pascal Bamford, PhD, Exact Sciences
Richard Bourgon, Freenome
Nate Carrington, PhD, Roche Diagnostics
Jonathan Chappell, Abbott
Kathy Culver, MS, GRAIL
Soheil Damangir, GRAIL
Seth Goldenberg, PhD, Veeva Systems
Stephen Hayward, Becton Dickinson
Andrea Johnson, Exact Sciences
Bryan Kamrath, JD, MS, Roche
Artie Kaushik, MBT, MBA, GRAIL
Christopher Keir, MD, MS, GRAIL
Patty Krantz-Zuppan, Medtronic
Martin Krockenberger, Exact Sciences
James Lowey, Tgen
Raphael Marcelpoil, PhD, Becton Dickinson
Tim McDaniel, PhD, Tgen
Magnus Miller-Willson, Beckman Coulter
Sam Mostafavi, Freenome
Daniel Nichita, MD, MS, Beckman Coulter
Christine Nichols, Hologic
Girish Putcha, MD, PhD, Freenome (Work Group Chair)
Nicholas Schork, PhD, Tgen
Erica Sethi, MS, Abbott
Brad Spring, Becton Dickinson
April Veoukas, JD, Abbott
Mohammed Wahab, MS, Abbott
Government:
Serafina Brea, MBEE, CMS
Sara Brenner, MD, MPH, FDA
Matt Diamond, MD, PhD, FDA
Manjula Gama Ralalage, MD, MS, CDC
Mark Del Vecchio, Tunnell/BARDA Contractor
Expert Advisors:
Susan Alpert, PhD, MD
Alberto Gutierrez, PhD
MDIC:
Zafar Azam, Program Manager, Clinical Diagnostics
Carolyn Hiller, MBA, Program Director, Clinical Diagnostics
