Objective: Identify and quantify FOG episodes on RGB, walkway pressure data and display the results in a web interface.

Background: Freezing of Gait is a clinical symptom that occurs in Parkinson’s disease
Traditional identification uses visual inspection, is time intensive[1], while ML algorithms focus on identification [2,3] and not on temporal segmentation. For pressure walkway data, Nantel et al [4] exists.

Method: 28 patients, suffering from FOG episodes underwent gait analysis (n loops of 5m), recording Movement from two angles: Front- and Back (30 FPS), pressure data using walkway hardware (120hz). Most patients underwent weekly assessments; maximum 9 visits per patient.
Each video is labeled by a physician, only retaining FOG videos. Temporal segmentation using AFSD algorithm[5], a state of the art algorithm for Temporal Action Localization. Training is done using the following settings (considered segment length:256, stride 10; 80% training, 20% testing). Considering the following inputs: raw RGB data, optical flow data (DualTVL) and a combination of them.
Walkway pressure data is processed according to Nantel et al [4], yielding an inferred start- and end-time per FOG segment.
Performance is defined as Map@IOU (Mean Average Precision at Intersection Over Union threshold).
An online platform is developed, identifying the patient using MaskRCNN+ SORT and a resnet152 network (pretrained on imagenet, finetuned on patient data).

Results: This resulted in 44 videos, 203 FOG episodes  (figure1).
[figure1]
AFSD on video data, achieves a comparable performance on higher MAP thresholds compared to the algorithm proposed by Nantel et al (table1 ), AFSD tends to have a better performance at higher threshold precision, but seems to underperform at lower threshold precision. A potential reason for this is an observed tendency of AFSD for false positive predictions and a tendency of the AFSD algorithm to confuse standing phases with FOG episodes.
[table1]
Prediction results are available through an interactive UI (picture 2, picture 3); allowing physicians to suggest corrections.
[figure2]
[figure3]

Conclusion: AFSD and temporal action localization algorithms hold promise to temporally segment FOG episodes, but reaching an acceptable standard needs additional research.
The algorithm by Nantel et al. is more sensitive, but precision can be improved. Combining the two modalities may enable more robust predictions.

References: References: 1. Cui C. & Lewis, S., Future Therapeutic Strategies for Freezing of Gait in Parkinson’s Disease. Front. Hum. Neuroscience, volume 15, 2021. 2. Hu et al. Graph Sequence Recurrent Neural Network for Vision-Based Freezing of Gait Detection. IEEE transaction on image processing, Vol 29, pp. 1890-1901, 2019. 3. Hu et al. Vision-Based Freezing of Gait Detection With Anatomic Directed Graph Representation. IEEE Journal of Biomedical and Health Informatics, vol. 24, no. 4, pp. 1215-1225, April 2020. 4. Nantel et al. Repetitive stepping in place identifies and measures freezing episodes in subjects with Parkinson’s disease. Gait & Posture, vol. 34, pp. 329-333, 2011. 5. Lin et al. Learning Salient Boundary Feature for Anchor-free Temporal Action Localization, CVPR, pp. 3320-3329, March 2021.

« Back to 2022 International Congress

MDS Abstracts - https://www.mdsabstracts.org/abstract/temporal-segmentation-of-fog-using-afsd-algorithm-on-rgb-data-and-on-walkway-pressure-data/