Objective: Assess the reliability and measurement error of measures derived from digital health technologies (DHTs) in Parkinson’s Disease (PD) patients and estimate sample size for disease modifying therapy (DMT) trials.

Background: DHTs are used in clinical trials to allow for collection of measures directly from patients, often with little burden, allowing for frequent assessment of the lived experience with PD.  The reliability and responsiveness to disease progression of these measures compared with in-clinic MDS-UPDRS measures remains undefined and directly impacts their use as an outcome measure in clinical trials, and these parameters strongly impact sample size calculations.

Method: We use publicly available data from Omberg et al [1] (8,550 assessments from 47 people, 24 with PD and 23 HCs, between 2015 and 2018) to compute finger tap measures using Koneksa algorithms [2]. ICCs from a mixed effects model were computed for individual and week-long bursts of assessments [3].  We extract the measurement error per measure as the residual variance from the model used to calculate ICCs. Data from clinical trials with a DMT that reduces the progression rate by 50% are generated according to a Gaussian state space model4 previously fit to MDS-UPDRS data.  DMT effect is determined by a statistically significant treatment arm x time interaction term in a mixed effects model.

Results: The ICC for test-retest reliability of individual assessments of tapping speed is 0.81 and of bursts is 0.92 [Table 1]. Using in-clinic MDS-UPDRS part III total scores taken at 3 months intervals, we would expect to need approximately 300 participants per study arm to achieve 80% power assuming an average increase of 2.63 points per year [4].  In contrast, with a digital measure that is equally responsive to underlying disease progression, we expect to need only 125 participants per arm completing weekly assessments or 100 completing monthly bursts of assessments [Figure 1].  

However, the responsiveness of digital measures to underlying disease progression is unknown.   The at-home assessments retain a sample size advantage so long as their responsiveness is > 60% of that of in-clinic measures.

Conclusion: Using digital assessments in clinical trials for DMTs in PD may reduce costs and shorten trials by allowing for treatment effect detection with fewer participants.

Table 1

Figure 1

References: 1 Omberg, L. et al (2022) Remote smartphone monitoring of Parkinson’s 
disease and individual response to therapy, Nature Biotech, 40 (4), 480-487.

2 Ellis, R., Kelly, P., Huang, C., Pearlmutter, A., & Izmailova, E. S. (2022). Sensor Verification and Analytical Validation of Algorithms to Measure Gait and Balance and Pronation/Supination in Healthy Volunteers. Sensors, 22(16), 6275. https://www.mdpi.com/1424-8220/22/16/6275

3 Ratitch, B., Trigg, A., Majumder, M., Vlajnic, V., Rethemeier, N., & Nkulikiyinka, R. (2023). Clinical Validation of Novel Digital Measures: Statistical Methods for Reliability Evaluation. Digit Biomark, 7(1), 74-91. https://doi.org/10.1159/000531054

4 Evers, L. J. W., Krijthe, J. H., Meinders, M. J., Bloem, B. R., & Heskes, T. M. (2019). Measuring Parkinson’s disease over time: The real-world within-subject reliability of the MDS-UPDRS. Mov Disord, 34(10), 1480-1487. https://doi.org/10.1002/mds.27790

5 Martinez-Martin, P., Rodriguez-Blazquez, C., Alvarez-Sanchez, M., Arakaki, T., Bergareche-Yarza, A., Chade, A., Garretto, N., Gershanik, O., Kurtis, M. M., Martinez-Castrillo, J. C., Mendoza-Rodriguez, A., Moore, H. P., Rodriguez-Violante, M., Singer, C., Tilley, B. C., Huang, J., Stebbins, G. T., & Goetz, C. G. (2013). Expanded and independent validation of the Movement Disorder Society–Unified Parkinson’s Disease Rating Scale (MDS-UPDRS). Journal of Neurology, 260(1), 228-236. https://doi.org/10.1007/s00415-012-6624-1

« Back to 2024 International Congress

MDS Abstracts - https://www.mdsabstracts.org/abstract/reliability-of-digital-measures-in-parkinsons-disease-and-their-implications-for-sample-size-calculations/