Objective: To quantify inter-rater agreement in independent clinician scoring of video-recorded MDS-UPDRS Part III assessments, and to characterize the distribution of scoring variability at both item and total score levels.

Background: Accurate and consistent motor assessment is fundamental to the clinical management and research evaluation of movement disorders. Although the MDS-UPDRS Part III is the gold-standard tool for quantifying motor severity, rater interpretations can vary, with agreement differing meaningfully across items [1,2]. This variability can introduce measurement error, affecting the precision of longitudinal tracking and the interpretation of treatment effects, particularly when assessments are conducted by different clinicians across sites or time points [3].

Method: Video recordings from a broader clinical study captured standardized motor assessments using a video-based motor assessment platform (PRIMS) (n=76 with PD). Two trained raters, independently and blinded to each other’s scores, rated a subset of MDS-UPDRS Part III items. Inter-rater variability was examined at the item level using exact agreement (identical scores) and near agreement (scores within one point).

Results: Exact agreement between raters ranged from 36.18% to 78.95% across items, with near agreement ranging from 81.94% to 99.18%. Agreement was highest for tremor at rest (exact: 78.95%, near: 99.18%), while exact agreement was lowest for left-hand movements (36.18%) and near agreement was lowest for facial expression (81.94%). Near agreement was substantially higher than exact agreement across all items, suggesting disagreements were typically minor.

Conclusion: Inter-rater variability in MDS-UPDRS Part III scoring was item-dependent, with notably lower agreement on facial expression and left-hand movements. These findings suggest that item-specific variability may meaningfully affect sensitivity to change and between-group comparisons in longitudinal and multi-site research. Future work should target rater calibration protocols and prospective multi-rater validation to improve scoring consistency across movement disorder assessments.

References: [1] Kenny, L., Azizi, Z., Moore, K., et al. Inter-rater reliability of hand motor function assessment in Parkinson’s disease: Impact of clinician training. Clin Park Relat Disord. 2024;11:100278.

[2] Richards, M., Marder, K., Cote, L., Mayeux, R. Interrater reliability of the Unified Parkinson’s Disease Rating Scale motor examination. Mov Disord. 1994 Jan;9(1):89-91.

[3] Turner, T. H., & Dale, M. L. Inconsistent Movement Disorders Society-Unified Parkinson’s Disease Rating Scale Part III Ratings in the Parkinson’s Progression Marker Initiative. Mov Disord. 2020;35(8):1488–1489.

« Back to 2026 International Congress

MDS Abstracts - https://www.mdsabstracts.org/abstract/inter-rater-variability-in-motor-assessment-scoring-measurement-implications-for-understanding-movement-disorders-in-aging/