Objective: To determine interrater agreement on the classification component of the Movement Disorders-Childhood Rating Scale (MD-CRS).

Background: The MD-CRS consists of three parts: classification, general assessment, and movement disorder severity. It was designed for use in children with mixed movement disorders.

Method: 112 videos of 66 children with hyperkinetic movement disorders were used in the study. The videos were a mixture of home videos and hospital videos. The videos were reviewed independently by five pediatric movement disorders neurologists, including a senior expert (Rater 1) and 4 individuals trained by him. For each participant the raters answered three questions: 1) Is more than one movement disorder present? 2) What is the [prominent] movement disorder? 3) What other movement disorders are present? If only one rater was an outlier, that individual was asked to reevaluate the video. Final interrater agreement was determined after that reevaluation. For each evaluation, Fleiss’ Kappa scores were computed to evaluate agreement. Cohen’s Kappa scores were used to determine the agreement between Rater 1 and the other raters. P-values were adjusted for multiple comparisons.

Results: The most frequent movement disorders in the sample were dystonia/athetosis (~30%), chorea/ballism (~30%), followed by myoclonus, stereotypy, tremor, and tic.  Rater 1 determined that 38% of the children had multiple movement disorders. Final agreement on number of movement disorders was 58%, with better agreement for individuals with only one movement disorder. For prominent movement disorder, the highest interrater agreement between any 2 investigators was 89% and the lowest was 76%, but absolute agreement among all five raters was only 42%. For other movement disorders, agreement ranged from 65% to 82% among any 2 investigators.

Conclusion: Interrater agreement ranged from poor to good, with highest agreement for the prominent movement disorder and the lowest for other movement disorders. We did not test the general assessment or movement disorder severity sections of the scale. For severity scores to be most meaningful, there must be high reliability in classification. Our results suggest that even with formal training, there is substantial disagreement among pediatric movement disorder’s experts. We suggest formal training to reliability on the MD-CRS prior to use in research studies.

« Back to MDS Virtual Congress 2020

MDS Abstracts - https://www.mdsabstracts.org/abstract/interrater-agreement-for-movement-disorder-classification-in-children-with-mixed-movement-disorders/