Objective: Our initiative aims to develop an integrated AR/MR platform for collecting diagnostic data and monitoring patients with neurodegenerative diseases using digital markers from registered signals.

Background: Neurodegenerative diseases, such as Parkinson’s disease (PD), present significant diagnostic challenges due to overlapping symptoms with other conditions and the absence of definitive tests. This complicates accurate diagnosis and effective treatment planning. Our research leveraged technological advances and data-driven methodologies to enhance patient care, focusing on the objective analysis of voice abnormalities in PD through spectral analysis and machine learning. This approach aims not only to improve diagnostic accuracy but also to offer insights into how different speech tasks can affect PD severity classification, potentially refining diagnostic techniques and patient outcomes.

Method: The process involves five distinct activities conducted with mixed reality eyewear, underlining the application’s compatibility with Microsoft HoloLens 2 for advanced feature extraction and AI-driven classification of PD. We analyzed five speech tasks within a broader 30-minute 17-task protocol recruiting 21 PD patients and 36 healthy controls. Utilizing the openSMILE tool and Azure Speech SDK, we conducted an in-depth analysis of acoustic and linguistic features, including fundamental frequency formant frequencies and language characteristics such as Part of Speech (POS) and Named Entity Recognition (NER). This comprehensive analysis contributes to our understanding of PD’s effects of PD on speech, and provides a foundation for future diagnostic and therapeutic strategies.

Results: The integration of AR/MR technologies in our study not only achieved a significant F1 score of 82.7 % %in identifying speech anomalies but also enhanced the patient experience by reducing the anxiety and discomfort often associated with traditional diagnostic exams.

Conclusion: This improvement in patient experience combined with promising diagnostic results underscores the potential of AR/MR technologies in revolutionizing the diagnosis and monitoring of neurodegenerative diseases. Looking forward, our research will continue to explore the scalability and real-world applicability of these technologies to further validate their impact and improve patient satisfaction.

The project was funded by The National Centre for Research and Development, Poland under Lider Grant no: LIDER/6/0049/L-12/20/NCBIR/2021

« Back to 2024 International Congress

MDS Abstracts - https://www.mdsabstracts.org/abstract/speech-processing-system-for-parkinsons-disease-using-mixed-reality/