Objective: To develop a multimodal framework fusing frontal-view video pose estimation frame coordinates with pressure-sensitive walkway features for parkinsonism detection, and to discover clinically meaningful PD gait subtypes from deep-learning embeddings.

Background: Gait impairment is a major symptom of Parkinson’s disease (PD), yet clinical assessment remains qualitative and rater dependent. Video-based pose estimation and pressure-sensitive walkways have each shown promise, but their combined diagnostic value and capacity to reveal disease heterogeneity remain unexplored.

Method: We analysed 1,417 patients (1,186 training; 231 prospective temporal hold-out) who underwent simultaneous frontal video recording and GaitRite walkway assessment. For video, 12 body-joint coordinates per frame were extracted (YOLO 11-L, RTMPose-M), yielding 80-channel time series classified by InceptionTime architecture (5-fold CV, 5-member ensembles). Walkway parameters (n=38) were classified with XGBoost model. Predictions were combined via optimised score fusion (0.6 video, 0.4 walkway). For subtyping, 256-dimensional InceptionTime embeddings from PD patients (n=886) were clustered (UMAP, KMeans) fit on training data; test patients were projected to the pre-trained space and assigned to nearest centroids.

Results: Fusion detection model achieved test AUC 0.772 (sensitivity 0.779, specificity 0.703, F1 0.746), outperforming video alone (0.738) and walkway alone (0.720) for patients with PD. Permutation importance identified ankle-to-ankle distance as the most predictive channel, followed by knee separation and arm-swing amplitude. Clustering of video embeddings revealed two PD gait phenotypes (silhouette=0.772), which had distinct cross-modality walkway measurement parameters: preserved-gait (74%; speed 85.0 cm/s, step length 47.3 cm) and impaired-gait (26%; 58.9 cm/s, 36.1 cm). Subtypes also generalized to the prospective hold-out set (test set silhouette 0.758).

Conclusion: Fusing frontal video keypoints with walkway measurements improves parkinsonism detection beyond either modality alone. Deep-learning embeddings reveal two PD gait phenotypes generalizing to a prospective hold-out, potentially aiding patient stratification in trials and serving as digital biomarkers for disease monitoring.

Video pose estimation

Classification performance and edge importance

PD gait subtypes and cross-modality validation

« Back to 2026 International Congress

MDS Abstracts - https://www.mdsabstracts.org/abstract/multimodal-gait-analysis-using-video-pose-estimation-deep-learning-and-walkway-parameters-parkinsonism-detection-and-phenotyping/