Objective: To evaluate whether a mixed-reality (MR) multisensor protocol combined with multimodal learning can discriminate Parkinson’s disease (PD) from healthy controls (HC) despite substantial missing data, and to compare early feature fusion with representation-level fusion.

Background: Early PD recognition is difficult because manifestations are heterogeneous and clinician-rated scales are partly subjective. MR-guided assessments can standardize task delivery while recording synchronized speech, kinematic, cognitive, and oculomotor signals.

Method: A cohort of 137 participants (66 PD, 71 HC) completed 17 MR-guided tasks spanning speech, hand/arm movement, tremor, gait, eye tracking, and cognition. From headset sensors, 4,850 numeric features were extracted and z-normalized. Missing values were handled by mean/median imputation, Multiple Imputation by Chained Equations (MICE), or variational autoencoders (VAE), including an attention-enhanced VAE. Model selection used 5-fold cross-validation on the training split (n=82), with final evaluation on a held-out set (n=55). Early fusion concatenated all features; representation fusion used modality- or task-grouped attention-VAE embeddings.

Results: Attention-VAE achieved the lowest error in masking experiments (RMSE ≈0.95), slightly better than MICE, while downstream classification was stable across imputation choices as shown on [figure1].Early fusion with Random Forest achieved 0.967 test AUC-ROC. Representation-level fusion remained strong but lower overall (test AUC-ROC ≈0.964 with modality-grouped and ≈0.945 with task-grouped embeddings). [figure 2] presents the comprehensive results. Feature-importance and ablation analyses highlighted speech and head-movement kinematics; removing head-movement features reduced test accuracy by ~0.31 and removing acoustic features by ~0.16 as presented in [figure3] and [figure4].

Conclusion: MR-based multisensor examinations coupled with robust imputation and multimodal fusion can support objective PD screening from incomplete recordings. Speech and head-kinematic measures appear most informative, suggesting potential for streamlining; larger, more heterogeneous cohorts are needed to confirm generalizability.

[figure1]

[figure2]

[figure3]

[figure4]

« Back to 2026 International Congress

MDS Abstracts - https://www.mdsabstracts.org/abstract/fusion-learning-with-attention-vae-imputation-for-mixed-reality-multisensor-screening-of-parkinsons-disease/