
Work Package 6
Clinical Demonstrations
Clinical Demonstrations
Objectives
Work Package 6 will demonstrate six clinical use cases. These use cases drive the development and implementation of the infrastructure to ensure that the infrastructure meets the needs of the health care work force and other users. The use cases will be the starting point for the implementation road map and facilitate design and development in co-creation with the clinical workforce.
First, an atlas of anonymous acute care cases will be established: MIMIC-EU (Use Case 1.) Furthermore, INDICATE will demonstrate improvement in the provision of care processes (Use Cases 5 and 6), and the development of new AI algorithms (Use Cases 2,3, and 4).
Healthcare use and research use
In addition, this work package will demonstrate that the infrastructure will address both a healthcare use of the data and a secondary use for the advancement of research and knowledge, meeting the functional needs of various stakeholders, including healthcare workers, (former) patients, researchers, and innovators like SMEs, industry partners, research institutes, and government entities as these stakeholders will actively participate in one or more use cases.
Virtual human twin
In short, the main objective is to demonstrate ICU datasets integration and work toward a “virtual human twin” of multi-scale, multi-disciplinary data to train computational models in at least four highly prevalent and high impact clinical domains, notably sepsis (infectious diseases), acute respiratory distress syndrome, acute kidney Injury, traumatic brain injury, and post- ICU syndrome.
Tasks
Task 6.1 aims to demonstrate the capabilities of the INDICATE platform for the aggregation of ICU datasets for secondary analysis and the development of clinically relevant AI algorithms by establishing an ’Atlas of anonymized Acute Care cases’ which we named MIMIC-EU. The MIMIC-EU workgroup will be coordinated by the SCO with contributions from experts in WPs 2, 3, 4 and 5. The data will include multiparameter physiological signals, laboratory test results, ICU procedures, medications, imaging reports, caregiver notes and clinical outcomes such as ICU- and in-hospital mortality, organ failure, organ support, length of stay etc. MIMIC-EU will further have the potential to be enriched with multi-modal data from several additional sources, including clinical notes from local electronic health records, clinical images adhering to EUCAIM Standards, and data from pre- ICU and post-ICU admission, as well as post-discharge information. Development will take an iterative approach, starting with the onboarding of mature and leading data providers (UDUS, CHU-Rennes) and continuously on- board more data providers thereafter. This approach will serve scientific purposes and will at the same time provide the backbone for the relevant deliverables and other use cases. This task includes:
- T6.1.1 Establishment of the formal MIMIC-EU workgroup that will collaborate with the Data Provider Support Office and the coordinators of Task T2.1 to define a minimal viable dataset and associated common data model and anonymisation requirements.
- T6.1.2 Drafting of template data access agreements and terms of use for data providers with the MIMIC- EU workgroup with the aid of experts from WP3 Task 3.4 ELSI framework
- T6.1.3 Integrate the MIMIC-EU atlas of anonymous care cases with the centralised metadata catalogue as a predefined selection across the federated infrastructure for ICU data.
- 6.1.4 Create a synthetic, fully anonymous dataset based on the full federated MIMIC-EU data to allow data users to test and validate data access requests and jobs prior to distribution and execution across the infrastructure.
- T6.1.5 Implement a secure workspace based on the templates developed in WP4 D4.1 where data users may bring their own tools or can utilise predefined tools.
- T6.1.6 Implement the security and quality policies and gates developed in WP4 to mitigate data protection risks and promote cyber security.
- T6.1.7 Create a set of straightforward analyses to showcase MIMIC-EU for promotion purposes. This will include analyses of important clinical scenarios and questions. Examples include a deeper analysis of hemodynamic monitoring in cardiogenic shock and sepsis, glucose metabolism in ICU patients, and regional differences in treatment strategies also in relation to resource allocation strategies. Furthermore, rare diseases will be analyses as well as treatment trajectories of old and very old patients and cancer patients. In these projects all partners of the academic hospitals in the consortium will be involved.
This task aims to demonstrate the development and deployment for clinical evaluation of machine learning models to predict organ failure based on time series data. The high availability of time series data is a unique selling point for ICU data. This use case will demonstrate how that data can be leveraged. To do so, this task will focus on the highly clinically relevant problem of (multi) organ failure. The premise being that if organ failure is predicted when still reversible, health outcomes would improve and clinical workflow would benefit as timely, planned interventions are more beneficial and pose less burden on health care workers. This task includes:
- T6.2.1 Select organ systems based on availability of data and outcome information from MIMIC-EU(T6.1) utilising the procedures, processes and templates developed in WP3 Task 3.4 ELSI frameworkand implemented in WP4 to gain access to data for federated analysis.
- T6.2.2 Establish a data adjudication committee to provide guidance on quality control and facilitate harmonised annotation of health outcomes, including organ failure of the following organ systems: kidneys, brain, liver, cardiovascular in collaboration with the Data Provider Support working group.
- T6.2.3 Develop, test, and validate machine learning models to predict organ failure of the specific organ system, taking into account regional differences, sex differences and minorities.
- T6.2.4 Design a clinical evaluation study for the machine learning systems.
- T6.2.5 Deploy the machine learning systems for clinical evaluation at participating institutions.
This task is dedicated to developing a virtual digital twin for critically ill patients, a process that involves numerous sequential decisions made every hour. This use case is particularly suited for reinforcement learning, which will be employed to determine the optimal treatment choices tailored to the specific conditions of individual patients at any given moment. A primary focus will be optimizing mechanical ventilation settings for treating respiratory failure. Crucial to this use case is the leverage of the MIMIC-EU dataset, as referenced in use case 1. The integration of this diverse and extensive dataset is key in customizing treatment decisions for a wide variety of patient profiles, thereby enhancing the generalizability of the developed models. Furthermore, supervised machine learning will be utilized to predict and prevent the need for reintubation and to guide clinicians on data collection for improved predictive accuracy. The outcomes of this work will be disseminated through scientific literature and a pre-print on the knowledge platform, facilitating the rapid exchange of best clinical practices across Europe. This task includes:
- T6.3.1 Utilise the procedures, processes and templates developed in (WP3 Task 3.4 ELSI framework ) and implemented in WP4 to gain access to data from the task contributors for federated analysis.
- T6.3.2 Develop and test a reinforcement learning model to optimise mechanical ventilator settings, taking into account sex differences, physiologic differences due to ethnic background, body mass index and comorbidities.
- T6.3.3 Develop guidance, training, and education materials on the development, testing and validation of reinforcement learning models in a federated setting for publication through the knowledge platform.
This task aims to validate and demonstrate the utilisation of the federated infrastructure for the special case of data access to data of paediatric patients. The additional complexity is in terms of the legal requirements of data protection for under aged data subjects across Member States. Current interpretation of data protection rules often results in data of paediatric patients not being shared at all, stifling innovation. Paradoxically, data sharing for research and innovation in paediatrics is vital, because populations of paediatric patients tend to be too small at individual institutions to enable research and in particular training and evaluation of machine learning models to support clinical practice. We will focus on sepsis at the neonatal and paediatric ICU, as sepsis remains the most common cause of neonatal (less than 28 days old) and children’s death. We will validate a promising, recently described early detection algorithm for sepsis that combines patient characteristics and bedside monitor data to generate a risk score for sepsis. Without large scale validation, implementing and adopting such an algorithm would not be possible. This task involves the following steps:
- T6.4.1 External validation of the early detection algorithm developed by Van Laere et al. ↗ We will assess the functionality of the algorithm in different neonatal populations, external validation is required in multiple datasets from different hospitals throughout Europe.
- T6.4.2 Refine the model to local settings. Even if results from external validation appear, it will be useful to assess the performance of the detection models after they have been retrained using all data from all patients as a training dataset. This effort will likely increase generalisability for application in different populations.
- T6.4.3 Set-up a continuous federated evaluation and retraining pipeline. After the early sepsis detection models are implemented, continuous evaluation of model performance is necessary to guarantee effectiveness and patient safety.
This task will demonstrate a use case of data sharing for the continuous improvement of clinical practice. High quality healthcare relies on consistent implementation of evidence for best practices, but achieving this consistency across diverse healthcare settings can be challenging. We will bridge this gap by creating a unified platform that allows hospitals to benchmark their practices against evidence-based data and each other. The Quality Benchmarking Dashboard will meet important clinical needs by offering real-time, customisable data access, potentially supporting evidence-based decision-making, and enabling quality control and improvement for various users and settings. It fosters research, cross-border collaboration, and patient-centred evidence- based care to enhance healthcare quality across Europe. To that end we will complete the following sub-tasks:
- T6.5.1 Identify clinically meaningful scenarios from scientific literature and clinical data, e.g. time- critical therapy, emergency response times, pain management, medical error detection.
- T6.5.2 Develop automated data processing pipelines using machine-readable transactions based on the ETL process of WP2 T3.
- T6.5.3 Create a user facing dashboard application that will utilise the infrastructure for automated data visualisation to compare pan-European clinical benchmarks to local patient data.
- T6.5.4 Create APIs that allows data users to integrate the clinical benchmark calculations into their own existing patient data management systems (e.g. Electronic Health Records)
- T6.5.5 Establish a consistent, anonymous benchmarking data set based on data disclosed through the federated infrastructure to reflect the heterogenic European population of the participating hospitals.
- T6.5.6 Generate a repository of machine-readable guideline recommendations for disclosure through the knowledge platform (WP5), in addition to the Quality Benchmarking Dashboard.
This task will develop a guideline for conducting Health Technology Assessments (HTAs) for all clinical demonstrations. The HTA systematically evaluates the characteristics and effects of the health technologies and interventions developed across the six use cases in the INDICATE project. The guideline will encompass an analysis of both intended and unintended consequences, offering evidence-based insights into the clinical, economic, social, and ethical aspects of these technologies. These assessments will be performed for each use case and will be included in the final report and/or scientific paper detailing the outcomes of each use case.
Lead

Christian Jung
Task leads

Alexandre Mebazaa

Paul Elbers

Rob Taal
