Category: Telemedicine
Objective: This study aimed to develop an immersive system using Microsoft HoloLens 2 for assessing motor and cognitive functions by integrating speech, movement, and eye-tracking analyses in neurodegenerative disease patients.
Background: Parkinson’s disease is a neurodegenerative disorder with symptoms such as tremors, bradykinesia, rigidity, postural instability, and speech impairments. MR technology allows convenient at-home data collection, minimizing stress. Combined with AI analysis, it creates an automated diagnostic system.
Method: Current assessments rely on subjective scales. MR and multimodal integration offer more precise evaluations. Our system combines speech, gait, hand movement, and eye-tracking analyses. The protocol includes 18 tasks guided by an MR application, lasting 27 minutes. 50 patients and 50 healthy controls participated. Clinical assessments included MDS-UPDRS, VHI, Tinetti Test, MMSE, and BDI. The system extracts 500+ parameters, such as speech (F0, jitter, shimmer), gait (step length variability), hand motion (3D tracking), and eye movement (saccade latency).
Results: Analysis showed significant differences in PQ sensor fidelity (p = 0.048) and SUS scores (p = 0.005), with large effect sizes (Cohen’s d = 1.16 and 0.78). The system successfully extracted multimodal parameters. MR assessments improved engagement and data quality. SUS scores confirmed high usability, with task performance correlating with clinical scales like MDS-UPDRS3, Timed Up and Go Test, and VHI.
Conclusion: This automated system enables detailed symptom assessment without physician supervision. MR enhances engagement and data precision, providing a valuable tool for clinical and research applications. The project was funded by The National Centre for Research and Development, Poland under Lider Grant no: LIDER/6/0049/L-12/20/NCBIR/2021 and by the Ministry of Science and Higher Education (MNiSW) under the project no. MNiSW/2025/DPI/53, Support for Students in Enhancing Their Competencies and Skills.
To cite this abstract in AMA style:
D. Hemmerling, J. Stepien, W. Szecowka, A. Channarayapatna Srinivasa, M. Rudzińska-Bar, M. Wojcik-Pedziwiatr, . . A Mixed Reality Multimodal System for Objective Assessment of Motor and Cognitive Functions in Neurodegenerative Diseases [abstract]. Mov Disord. 2025; 40 (suppl 1). https://www.mdsabstracts.org/abstract/a-mixed-reality-multimodal-system-for-objective-assessment-of-motor-and-cognitive-functions-in-neurodegenerative-diseases/. Accessed October 5, 2025.« Back to 2025 International Congress
MDS Abstracts - https://www.mdsabstracts.org/abstract/a-mixed-reality-multimodal-system-for-objective-assessment-of-motor-and-cognitive-functions-in-neurodegenerative-diseases/