Objective: To validate a novel multimodal mixed-reality (MR) platform utilizing HoloLens 2 for the automated assessment of Parkinson’s disease (PD) symptoms and to determine the system’s usability and immersion levels across patient and control groups.

Background: A complete neurological evaluation requires assessing motor control, gait, and gaze stability alongside speech. Immersive technologies offer a unique opportunity to create a standardized, stress-free environment that captures a broad spectrum of bio-signals, potentially mitigating patient anxiety associated with traditional clinical settings.

Method: The study included 47 participants (15 PD patients, 32 healthy controls) who underwent a 30-minute automated protocol comprising 17 structured tasks. Unlike single-modality studies, this system utilized the full array of head-mounted sensors (cameras, depth sensors, IMU, microphones) to assess saccadic eye movements, hand tremors, gait (Timed Up and Go), and cognitive responses. User experience was rigorously evaluated using the Presence Questionnaire (PQ) to measure realism and sensor fidelity, alongside the System Usability Scale (SUS).

Results: Quantitative analysis revealed high immersion levels, with the PD group achieving a mean PQ score of 5.28, confirming strong environmental presence comparable to healthy controls 5.88 However, distinct usability challenges emerged: while controls rated the system as “excellent” (SUS: 80.23), the PD cohort reported significantly lower usability (61.25, p=0.005). Notably, the perception of “Sensor Fidelity” showed a statistically significant difference between groups (p=0.048), suggesting that motor impairments may influence how patients perceive interaction accuracy.

Conclusion: The mixed-reality system successfully creates a high-immersion environment that encourages patient autonomy across a wide range of diagnostic tasks. While the concept is validated by high presence scores, the disparity in usability metrics highlights the need for specific interface adaptations for neurodegenerative cohorts. The platform demonstrates feasibility for holistic, multimodal remote monitoring, extending beyond simple voice analysis.

« Back to 2026 International Congress

MDS Abstracts - https://www.mdsabstracts.org/abstract/holistic-sensor-based-profiling-of-parkinsons-disease-via-mixed-reality-implementation-and-multimodal-feasibility/