iWe have been writing about software as a medical device (SaMD) for years, tracking the Food and Drug Administration's (FDA) efforts to keep up with the fast-paced development of digital technology, such as launching the Digital Health Center of Excellence, implementing predetermined change control plans, and issuing various digital health guidances on device software functions, clinical decision support software, cybersecurity, and other topics. In anticipation of FDA’s Artificial Intelligence /Machine Learning (AI/ML) Medical Devices Workshop in October 2021, we posted a brief history of the agency’s regulatory oversight of software through the traditional medical device regulatory framework established in the 1970s, in which we highlighted the numerous challenges associated with such an approach. But now, with the rise of artificial intelligence and machine learning and the proliferation of AI/ML-enabled software throughout the health care industry, FDA is facing enormous challenges using an outdated, procrustean regulatory framework to maintain standards of safety and quality for such software devices. It is becoming increasingly clear that innovation in the AI/ML and digital health technology space is advancing rapidly, as FDA Commissioner Rob Califf has emphasized in many recent public appearances, and that the traditional device framework is quickly becoming unworkable for such technologies.
What Are the Challenges with the Current Medical Device Framework?
In general, FDA has applied the same regulatory standards, including device classification, authorization pathways, marketing submissions, and quality requirements, to both hardware and software devices. The agency has little choice in the matter because the Federal Food, Drug, and Cosmetic Act (FDCA) does not provide for separate regulatory pathways or legal requirements for SaMD. As a result, FDA is forced to figure out ways to adapt the existing system as much as possible within the statutory authorities granted by the U.S. Congress to maintain a semblance of regulatory oversight over modern software devices.
Lack of a separate regulatory framework, risk classification system, or premarket review method for SaMD has resulted in many challenges for FDA, especially with respect to SaMD based on AI/ML algorithms that can change in real time as they process data. Before 2020, FDA fit AI/ML algorithms into the traditional device process by requiring the manufacturers to lock the algorithms after training them on a cultivated data set and before product authorization and commercialization. Locking the algorithms prevented their self-modification in the field, because existing regulations require the manufacturer to specifically implement all modifications and consider whether such modifications require separate FDA clearance or approval. Realizing that requiring locked algorithms for SaMD was not a permanent solution, FDA started authorizing full AI/ML algorithms with predetermined change control plans (PCCPs) in 2021, and published a draft guidance on developing such plans and including them with marketing submissions in April 2023. However, utilization of a PCCP only addresses the specific post-market issue of algorithm self-modification; there is still no regulatory framework for minimizing the risk of incorporating bias into the algorithm at the design and training stages, testing AI/ML algorithm performance prior to marketing authorization, or tracking post-market performance and patient outcomes.
Why Can’t FDA Use a Different Regulatory Pathway for AI/ML-Enabled SaMD?
In short, the FDCA establishes only three pathways for a medical device (other than class I or other exempt devices) to obtain marketing authorization: premarket notification (Section 510(k)), premarket approval (Section 515), and De Novo classification (Section 513(f)(2)). The FDCA therefore does not grant FDA the authority to develop and implement new authorization pathways for any type of medical product. Interestingly, FDA explored a potential alternative to product-focused marketing authorizations for software devices when it conducted the Software Precertification Pilot Program, which allowed participants to demonstrate robust quality systems and commitments to organizational excellence in order to streamline the agency’s regulatory review of their SaMD products. The pilot ended in September 2022, and the agency admitted that legislation would be necessary to implement a new paradigm for regulatory oversight of medical device manufacturers and their products.
What Could FDA Do Without Legislation Authorizing a New Regulatory Pathway?
It is highly unlikely that Congress will enact legislation granting FDA new authorities to implement additional regulatory pathways or substantially modify device regulations in other ways. Even without such statutory changes, however, FDA has some flexibility to build new requirements into the traditional device framework. For example, the agency has almost total discretion over the types of information a sponsor must include in a regulatory submission for marketing authorization. In addition, FDA has general authority to impose post-market requirements upon granting marketing authorization, including requirements to conduct additional clinical trials or establish patient registries to monitor device performance and outcomes. FDA could employ these methods to increase the pre- and post-market requirements on AI/ML-enabled SaMD manufacturers towards the ultimate goal of helping ensure adequate quality controls and patient safety.
As one example, FDA could require manufacturers to perform extensive pre-market testing of AI/ML-enabled SaMD against models that use real-world data and demographic information. Although the safety and efficacy of a class III device must be demonstrated in a pivotal clinical trial before receiving FDA approval, class II and non-exempt class I devices often require non-clinical performance testing to obtain marketing authorization. Even when a clinical trial is required, subject populations often lack diversity and may not adequately test the performance of an AI/ML algorithm or identify latent inherent biases. To supplement the current testing obligations, FDA could require manufacturers to perform in silico testing against real-world models with actual anonymized data to validate algorithm performance and help identify any performance issues or weaknesses. The agency can simply make such testing a requirement for any type of submission for marketing authorization of a AI/ML-enabled SaMD product.
Another incremental regulatory control that could be imposed is one or more human factors studies to determine how human operators use an AI/ML-enabled SaMD in clinical conditions with the attendant pressures of the clinical environment. FDA often expresses concern that health care professionals may simply trust clinical outputs from a SaMD algorithm that is a non-transparent black box without looking for concurrence in other information sources or alternative diagnostic or treatment methods, even though the SaMD output may be inaccurate. Requiring device manufacturers to perform human factors testing with actual health care professionals in a simulated clinical environment would help identify such issues and allow manufacturers to address them through special controls, such as training for the clinicians or specified disclosure requirements in connection with the SaMD output.
As a final example, FDA could implement post-market performance and safety data monitoring and reporting for AI/ML-enabled SaMD, which would help ensure that both the manufacturer and agency are aware of and can promptly address emerging issues with the product. The complex and changeable nature of AI/ML-enabled SaMD, even those under a PCCP, means that constant monitoring of performance in the field is necessary to identify risks and ensure safety, especially for diagnostic and therapeutic SaMD involving higher risks for users or patients. FDA is also working to develop the National Evaluation System for health Technology (NEST), which will eventually enable performance and patient outcome tracking across traditional devices and SaMD, but implementation of such system is likely still years away.
Conclusion
The push to expand the use of AI/ML software that is capable of self-modification during the delivery of health care has already begun, and it is becoming increasingly clear that the current device regulatory framework is not well-suited to ensure that the risks entailed in the development, validation, and use of such software are effectively identified and mitigated. This is not FDA’s fault given that the FDCA limits what the agency can do to evaluate and authorize SaMD products. In the past, FDA has developed creative solutions using its existing device statutory authorities to enable the review of risks associated with standalone software devices (e.g., evolving cybersecurity requirements and PCCPs). We expect that FDA will be able to further adapt to the quickly evolving field, by implementing additional testing, submission, and post-market requirements for AI/ML-enabled SaMD, such as the ones we suggest above. However, we hope that Congress will act at some point to give FDA additional authority to classify, authorize, and regulate AI/ML devices in a way that fits the technology, enables and incentivizes innovation, and enhances patient safety.
