The evolution of AI in healthcare is happening at such a rapid pace in some areas that it is often hard for physicians and their practice staff, to keep up with it. This goes for laws and regulations directed toward AI as well.
“It is really important for physicians to remember where we are right now in terms of the legal framework for AI, which is that we are in a state of shifting sands. There are emerging laws and regulations and at this point it is not exactly clear to physicians…what they need to do to use AI tools in a legally compliant manner,” said Kathleen Healy, Esq, a partner at Robinson & Cole in Boston, who advises clients on healthcare regulatory matters.
Evolving Federal and State Rules
At the federal level, Healy cited the final rule issued in January 2024 by the Office of the National Coordinator for Health Information Technology in the Department of Health and Human Services. The rule, which went into effect July 5, 2024, created a new regulatory framework for AI developers of certified health information technology. However, she noted that it also includes requirements that apply to “covered entities,” which include healthcare providers.
“If you actually go through the regulations, covered entities will need to comply with certain prohibitions on discrimination by May 1, 2025,” said Healy. “The final rule highlights a challenge for physicians and practices because they might look at the regulations and think they only apply to AI developers of certified health information technology.”
Healy also said that a number of bills are being introduced at the state level.
Valerie Rogers, senior director of government relations with the Health Information and Management Systems Society (HIMSS), confirmed that, “The majority of the state legislation passed or proposed is focused on creating advisory bodies and task forces to define, seek transparency, eliminate bias in the application of AI, and in a few cases to regulate uses in health.” Alabama, Colorado, Illinois, and Vermont have passed such laws. In addition, Alabama, Colorado, Illinois, and Mississippi have passed bills that execute limitations on the use of AI in their states, but many of these bills are not health-specific.
“There has been an explosion of interest, models in healthcare, and guidance that legislators are taking notice of,” said Rogers. “Many doctors are already using AI in healthcare, and there are a growing number of projects that are using it to generate rapid, actionable insights from data to inform policies, public health interventions, and emergency response.”
Rogers cautioned, however, that a major trend at the state level is “inconsistent knowledge” of AI in general and how it applies to healthcare. She added, “In some cases, state policymakers are drawing from anecdotal evidence via reports from other state legislators and stories from the media and some social media such as X and LinkedIn.”
Rogers has also found that some lawmakers are struggling to determine the difference between generative AI and conversational AI; to make it more challenging, the way states and localities refer to these terms varies.
“We anticipate a shift in trends as state officials learn more about the benefits and potential drawbacks of AI in healthcare and public health [including], administrative applications such as billing, scheduling, or notetaking versus clinical applications that are less understood,” said Rogers.
Guidance from Organizations
To assist with the development and implementation of AI, HIMSS has released a set of global policy principles on it and machine learning, which Rogers encourages lawmakers to review when considering drafting legislation. The principles cover a number of issues, including policy guardrails, mitigating bias, and protecting patient data. They also reiterate the crucial role of human supervision of AI.
“HIMSS AI Global Policy Principles illuminate the fact that AI, no matter how exact the model purports to be, will always need human judgment and discretion (including ensuring not only accuracy but making sure that the data, analysis, and results are not biased),” said Rogers.
The American Medical Association (AMA) has created a framework for developing and using AI as well. It envisions a healthcare system where AI enhances patient care, improves population health, reduces costs, and improves the work lives of providers and lays out ethical checkpoints for developers, deployers, and clinicians to ensure that they are meeting those goals.
Risks to Be Aware Of
In implementing an AI tool to enhance their practice’s operations, Healy encourages physicians to be mindful of certain legal risks. These include data privacy and cybersecurity issues that can occur because AI tools require large amounts of data, often including health information that is protected by HIPAA.
In addition, she said healthcare providers should be proactive in addressing potential bias and discrimination that can occur because AI tools’ training data is often based on a certain subset of the population. Another issue is the unlicensed practice of medicine, where a physician who is using an AI tool relies on its results solely without using their own clinical judgment to check the results.
“One example that I often give is there are a lot of radiology tools that use AI, and they can assist radiologists to interpret images and analyze large volumes of data. The risk, however, is that if you are utilizing those tools and replacing the clinician's judgment, it can result in a claim for the unlicensed practice of medicine,” said Healy.
In outlining these aspects to factor in, Healy said she does not want to discourage practices from exploring using AI. Rather, they should bring in legal experts to guide them.
“It’s a technology, so you want someone with expertise in technology laws and contracts. The laws will also continue to get more and more specialized very quickly, so you also don't want somebody who is starting from scratch,” she said.
