How do we know whether digital health interventions work as advertised?

Background: As digital health interventions proliferate, the challenge of rigorously evaluating their real-world effectiveness becomes increasingly crucial. Traditional clinical validation methods often prove inadequate for rapidly evolving digital solutions, yet robust evidence remains essential for ensuring patient safety and therapeutic value. This focus topic day explores innovative approaches to validating digital health interventions, examining the tensions between rapid technological advancement and thorough efficacy assessment. By addressing questions of appropriate evaluation frameworks, real-world evidence collection, and the impact of personalization in continuous performance monitoring, we aim to advance the discourse on establishing trusted, evidence-based standards for digital health validation while fostering innovation.

Chair: Dr. Stefan Tino Kulnik, MRes

Speaker:

• Dr. Anna Eleonora Carrozzo
  Post-Doc in Statistics | Salzburg Research and LBI-DHP

Abstract:

 N-of-1 trials are currently receiving broader attention in healthcare research when assessing the effectiveness of digital interventions. In such approaches, a single subject undergoes a sequence of interventions (typically, an experimental versus a control condition) in a random order to identify the best personalised strategy. Moreover, when a series of N-of-1 trials are properly aggregated, it becomes possible to detect an intervention effect at a population level. In this talk, we will investigate whether a meta-analysis of summary data from a series of N-of-1 trials can help detect a statistically significant intervention effect with fewer participants than in traditional, prospectively powered two-arm RCT and crossover designs.

Speaker Bio:

• Eleonora Carrozzo is a post-doc researcher within the inter-institutional Excellence in Digital Sciences and Interdisciplinary Technologies (EXDIGIT) project between the Salzburg Research Forschungsgesellschaft (SRF) and the Paris Lodron University Salzburg (PLUS). Her background is in statistics, in particular in the field of nonparametric statistics and resampling techniques. She has many years of expertise in applied statistics in the healthcare and life sciences, which she also gained during her prior work at the Ludwig Boltzmann Institute for Digital Health and Prevention, where she held a post-doc position for statistics in digital health trials. 

Speaker:

• Prof. Rik Crutzen
  Professor of Behaviour Change & Technology and Head of the Department of Health Promotion | Care and Public Health Research Institute | Maastricht University, The Netherlands

Abstract:

Evaluation does not happen at the end of an intervention trajectory, but starts during initiation of ideas. Intervention Mapping is a systematic approach to guide intervention developers in the process of developing an adequate evaluation plan. This is helpful to interventions in general and in this talk there is a specific focus on digital behaviours change interventions. Evaluations of these interventions come with their own challenges and opportunities. In this talk, there will be attention to methodological challenges and opportunities as well as (the evidence needed for) the impact of these intervention on individual patients and the society at large.

Speaker Bio:

• Rik Crutzen is a Professor of Behaviour Change & Technology and Head of the Department of Health Promotion, Care and Public Health Research Institute, Maastricht University, the Netherlands. He is also affiliated to the Ludwig Boltzmann Institute for Digital Health and Prevention. A central theme in his work is the systematic development, implementation, and evaluation of behaviour change interventions (i.e., using and developing the Intervention Mapping approach). He has a strong focus on methodological and theoretical progress.