The conversation we are not really having
When health systems talk about AI right now, the questions tend to circle around speed, cost, and competitive pressure. Will we fall behind? Are we deploying fast enough? What is the ROI? All fair questions, honestly, and I get why they dominate the room.
But there is another question, quieter, that I keep coming back to: who is this AI actually working for, and who is it quietly working against? Because the data so far suggests that when you deploy a model trained on populations that look one way into a patient population that looks another, you do not get a neutral outcome. You get amplified disparities, dressed up as objective decision support.
KFF recently laid this out in a way I think every healthcare executive should sit with for a few minutes. Experts in health, medicine, technology, and policy are calling for ongoing dialogue and ethical commitment from all stakeholders, and the Council of Medical Specialty Societies along with the Doris Duke Foundation have already created an alliance, Encoding Equity, specifically to address bias in AI deployment across health systems. That alliance does not exist because everything is fine.
The reframe: equity is not a compliance checkbox, it is a deployment strategy
Here is what I want to invite leaders to consider. Most of the conversation around AI bias gets framed as a regulatory concern, something the legal and compliance team will eventually flag (and then a working group will be formed, and a policy will be written, etc). That framing is not wrong, but it is incomplete, and I think it is actually holding organizations back.
Equity in AI deployment is a clinical quality issue. It is a workforce trust issue. It is a patient safety issue. And in my experience, when you treat it as a compliance checkbox, you end up with documentation that satisfies an auditor and a tool that still under-serves a meaningful portion of your patient panel.
The reframe I would offer: the organizations that take equity seriously up front are not slowing down their AI strategy. They are protecting it. Because a model that quietly performs worse for 30% of your patients is not a deployment success, it is a liability waiting to be discovered (and it usually gets discovered the hard way).
What the evidence is telling us
The 2023 RAISE international symposium, which KFF references, brought together a pretty wide coalition of voices on exactly this question: how do we make sure the AI being deployed across health systems is inclusive, and not just inclusive in the marketing materials, but actually in the training data, the validation, and the monitoring after go-live? The fact that this needed to be a symposium, and that the call coming out of it was for ongoing dialogue rather than a one-time fix, tells you something about where the field actually is.
Meanwhile, on the workforce side, Healthcare Executive made a point that I think connects to all of this even though equity is not the headline. Leaders at organizations like Ochsner are quick to remind everyone that even with all the impressive technology, human beings are still very much the decision-makers in the clinical relationship. That matters here, because if your clinicians do not trust the tool, or if they suspect it does not perform equally well across their patient panel, they will work around it. Which means your investment underperforms regardless of how clever the model is.
And then there is the upskilling angle. AstraZeneca has trained 17,000 employees as AI-certified, tying it directly to an $80 billion stretch revenue goal. That is a serious investment, and it is one that I think gets the order of operations right: the people first, then the tools. Because if your workforce does not understand how a model can fail, or who it can fail for, they cannot catch the failure when it happens (and it will happen, that is just part of the learning, it’s messy).
The Encoding Equity alliance, the RAISE symposium, the workforce trust signals, the upskilling investments at companies like AstraZeneca, all of this is pointing in the same direction. Equity is not separate from AI strategy. It is the substrate underneath it.
What this means for healthcare leaders
If you are running a health system, or sitting in a CHRO or CMIO seat, I want to validate something first: the pressure to move fast on AI is real. The board is asking. The competition is moving. Vendors are calling every week (a lot of $$ flowing through that pipeline right now). I am not suggesting you slow down for the sake of slowing down.
What I am suggesting is that the questions you ask before deployment matter more than the speed of deployment itself. A few that I would put on the table:
How was this model trained, and on which populations? If your patient demographics differ meaningfully from the training population, that is not a dealbreaker, but it is something you need to know going in (and actually listening to the answers, not just collecting them).
How is performance being monitored across patient subgroups after go-live, not just in aggregate? Aggregate accuracy can look great while subgroup performance is quietly underwater.
Who on your team is equipped to interpret these answers? This is where the workforce piece comes back. You cannot govern what your people are not equipped to evaluate, and given what I have seen so far, most health systems do not yet have that capability spread widely enough.
And maybe the hardest one: what is your plan when you find a problem? Because you will find problems. The organizations that handle this well are the ones that decided ahead of time how they would respond, rather than scrambling when an issue surfaces.
What becomes possible
I want to close on a more hopeful note, because I think it is warranted. The healthcare leaders I talk with who are taking equity seriously in their AI deployments are not doing it because they are worried about regulators (or at least, not only that). They are doing it because they see an opportunity that most of their peers are missing.
When you build equity into your AI deployment from the start, you end up with tools that perform more reliably across your full patient panel. You end up with clinicians who trust what they are working with, which means actual adoption rather than the polite kind that shows up in survey data but not in workflow. You end up with a workforce that is genuinely more capable, because you invested in their understanding rather than just their compliance training.
And, honestly, you end up with an AI strategy that holds up over time. The shortcuts being taken right now in the name of speed are going to be visible in two or three years, and the organizations that took the longer path are going to look like they were ahead all along.
Given what I have seen so far, the leaders who are willing to ask the harder questions now, the ones about who the technology actually serves and who it might be quietly failing, are the ones building something durable. Not a faster horse. Something more like a better foundation. And in healthcare, where the stakes for getting this wrong are not abstract, that foundation is worth taking the time to build.
