AI in Healthcare: ‘Often seen as a band-aid for structural problems’

Recently, INDICATE co-lead Michel van Genderen shared his expertise on the Dutch Radio national news program De Nieuws BV (BNN VARA) at NPO Radio 1, discussing the challenges of implementing artificial intelligence in healthcare. 

Artificial intelligence is often presented as the solution to the growing problems in healthcare. From administrative burdens to staff shortages, in theory, AI could provide relief. But in practice, only two percent of the AI applications studied are actually used on the care floor. “We conducted this research in 2021 as well, and we see exactly the same outcome now,” says internist-intensivist Michel van Genderen on De Nieuws BV. “We simply aren’t making any progress.”

According to the internist, the problem doesn’t lie in a lack of innovation, but in how that innovation is implemented. AI is often applied on top of existing healthcare systems without fundamentally changing those systems. AI is used because it can be used, not because it actually solves a real problem. “Healthcare is like a car,” Van Genderen explains. “And what we’re doing now is trying to install all these new gadgets, while the car itself actually needs to be redesigned. Many AI models are technically very impressive, but in practice they don’t solve a relevant, existing problem.”

---

“Predicting models can perform very well based on a dataset, but the real question is: what use is it for a doctor or patient? What problem are we actually solving with this?”

---

Clinical Application

This is evident, for example, in AI models that predict the likelihood of a patient dying. “Predicting models can perform very well based on a dataset,” says the internist. “But the real question is: what use is it for a doctor or patient?” Without a clear clinical application, the added value of these kinds of tools remains limited. The fundamental question “What problem are we actually solving with this?” is, according to Van Genderen, too rarely asked when developing AI applications.

The AI Hype

The AI hype is reinforced by consumer behavior. Technologies like ChatGPT are adopted rapidly, often without critical questions about reliability. This pattern can also be seen in healthcare. “We tend to think: this is new and smart, so we should use it,” says the internist. But in healthcare, different standards apply: transparency, safety, and explainability are crucial. “AI sells well and sounds promising, but it doesn’t always provide good solutions.” Discussions about responsibility are quickly seen as a hindrance to progress, whereas that responsibility is actually essential.

Innovation is Necessary

This nuance is important because the problems in healthcare are real. Van Genderen points to his experiences during the COVID-19 pandemic: the demand for care is growing rapidly, while the number of healthcare workers is falling behind. Technological innovation is therefore necessary. According to Van Genderen, AI is too often seen as a band-aid for structural problems. “Some of the things we try to solve with AI might not even need solving,” he suggests.

Ethics by Design

Recent research shows that language models respond differently in medical diagnoses when sociodemographic characteristics are included. In some cases, this leads to a completely different, and sometimes incorrect, assessment. Such “black-box” models undermine the trust of healthcare professionals and patients. Van Genderen argues that trust is essential for successful innovation. That’s why he advocates for so-called ethics by design – ethical, legal, and societal considerations that must be incorporated from the very start of the development process. “If you do it afterward, it’s already too late,” he states.

Read the full article or watch the radio item here (in Dutch).

Source: De Nieuws BV NPO Radio 1