Automatic segmentation of demyelination lesions in multiple sclerosis

Alexander V.  Zakharov; Захаров А. В.; Igor V. Shirolapov; Широлапов И. В.; Elena V. Khivintseva; Хивинцева Е. В.; Mariya S. Sergeeva; Сергеева М. С.; Natalya P. Romanchuk; Романчук Н. П.; Dmitry A. Dedyk; Дедык Д. А.; Darya D. Melnikova; Мельникова Д. Д.; Arseniy M. Andreev; Андреев А. М.; Alexandra I. Mavletova; Мавлетова А. И.; Anton O. Shchepetov; Щепетов А. О.; Jude Hemanth; Hemanth Jude

doi:10.35693/SIM636947

Automatic segmentation of demyelination lesions in multiple sclerosis

Authors: Zakharov A.V.¹, Shirolapov I.V.¹, Khivintseva E.V.¹, Sergeeva M.S.¹, Romanchuk N.P.¹, Dedyk D.A.¹, Melnikova D.D.¹, Andreev A.M.¹, Mavletova A.I.¹, Shchepetov A.O.¹, Hemanth J.²
Affiliations:
1. Samara State Medical University
2. Karunya Institute of Technology and Sciences
Issue: Vol 9, No 4 (2024)
Pages: 284-290
Section: Medical Informatics
Published: 15.12.2024
URL: https://innoscience.ru/2500-1388/article/view/636947
DOI: https://doi.org/10.35693/SIM636947
ID: 636947

Cite item

Full Text

Abstract
Full Text
About the authors
References
Supplementary files
Statistics

Abstract

Aim – to evaluate the effectiveness of the YOLOv8 algorithm for automatic segmentation of demyelination lesions in various locations in patients with multiple sclerosis.

Material and methods. The study included 120 patients with a clinically confirmed diagnosis of multiple sclerosis who underwent contrast-enhanced MRI. The MRI data from patients with different types of disease progression were analyzed. T1-weighted, T2-weighted, and FLAIR sequences were used for the analysis. The YOLOv8 algorithm was adapted for medical imaging and trained on manually annotated MRI scans. Model performance was evaluated using precision, recall, and F1-Score metrics.

Results. The YOLOv8 model demonstrated high segmentation performance with a precision of 0.79, recall of 00.73, and F1-Score of 0.65. The model effectively identified demyelination lesions in various locations typical for multiple sclerosis. However, there remains a need to improve recall to minimize the missed lesions. Testing on independent data confirmed the stability of the results of the model.

Conclusion. The YOLOv8 algorithm shows significant potential for automatic segmentation of demyelination lesions in multiple sclerosis patients. This method could be successfully implemented in clinical practice, enabling faster diagnosis and improved monitoring of disease progression. Further optimization of the model, through data augmentation techniques and hybrid architectures, may enhance both segmentation accuracy and recall.

Keywords

magnetic resonance imaging, multiple sclerosis, segmentation, deep learning

Full Text

Multiple Sclerosis, or MS, is a neuroinflammatory disease of the central nervous system (CNS) causing demyelination and neural damage. The disease brings about frequent disability among young people (aged 18-40) [1]. The progression of disability in the MS patients significantly affects social, economic and social well-being. The annual economic burden of MS in the USA is 85 Billion US Dollars. Similar data was obtained in the European Union, where average annual expenditure varies from 22,800 Euro (mild degree of the disease) to 57,500 Euro (severe degree of the disease) in terms of purchasing power parity, the direct medical expenses making up to 68% of the total expenditure [2]. Timely diagnostics is an important factor in decreasing the disability caused by the disease by early prescription of disease-modifying therapy (DMT) [3].

The diagnosis of MS is based on the 2017 McDonald diagnostic criteria [4], which provide increased diagnostic accuracy for this disease based on clinical, imaging and immunological indicators. The main thesis of these criteria is the detection of dissemination of clinical or instrumental signs in space and/or time [4].

Dissemination in space is characterized by the emergence of foci of demyelination in the following anatomic regions of the CNS: periventricular region of the brain, cortical or juxtacortical region of the brain, infratentorial region of the brain or the spinal cord, which indicates multifocal damage. Dissemination in space may be manifested by one or several T2-hyperintensive foci characteristic of the MS, at least in two of the four regions of the CNS [5]. Dissemination in time describes the development or appearance of new foci of demyelination in the CNS over time.

A key tool for diagnosing and monitoring patients with MS and a major component of the continually updated diagnostic criteria for MS is MRI [6].

The process of identification and segmentation of lesions in MS is usually performed manually by skilled neuroradiologists; it is a labor-intensive task prone to error [6]. Therefore, a necessity exists to develop automated tools to facilitate the procedure.

At present, work on automation of segmentation of demyelination foci using machine-learning algorithms has yielded results that are impressive enough to begin implementing them into routine clinical practice [7, 8].

Many other automated methods have been developed for damage detection and segmentation. Methods such as ‘k-nearest neighbor’ [9, 10], Support Vector Machines (SVMs) [11, 13], Markov random fields [14, 15], ‘random forest’ [16, 17] or special algorithms based on intensity [18-21]. The category of deep high-level neural networks prevailed, that were most frequently represented by convolutional neural networks (CNN) in the form of U-nets [22]. A modification of this network was proposed, the nnU-Net [23], a method that automatically sets up pre-processing stages, architecture, training and post-processing for improved adaptation to the properties of the data set and available equipment.

Thus, in the majority of present studies, the automatic segmentation of the MRI data uses deep learning algorithms with the U-net architecture [24] in its 2D and 3D modifications [25].

Despite the continuously accumulating MRI data, it has not yet been possible to effectively solve the task of obtaining quality and stable metrics of segmentation algorithm performance for them to be used in clinical practice. Therefore, individual groups of researchers continue looking for the optimal architectures of neural networks, pre-processing of MRI data, synthetic data [26], and use of several architectures or approaches for their training [27, 28].

Identification and segmentation of the new foci of MS remain highly challenging tasks. At present, automatic methods may be more sensitive for the identification of new lesions but yield more false positives when compared to manual segmentation by radiologists experienced in evaluating the MS patients’ MRI data [8].

AIM

Use the new deep learning algorithm YOLOv8 to identify demyelination foci in the subcortical, infratentorial, periventricular and juxtacortical localizations, and to calculate the volume of the same.

MATERIAL AND METHODS

The study was approved by the Ethics Committee of the Federal State Budgetary Educational Institution of Higher Education “Samara State Medical University” of the Ministry of Health of the Russian Federation (Protocol No. 52 dated 12.12.2023).

The study included 120 patients with active stable progression of the MS diagnosed in compliance with 2017 McDonald diagnostic criteria [4]. The average age of patients was 35.7±10.2 years, and the gender distribution was even. The disease activity was established based on the availability of one exacerbation within the past year or two exacerbations within the past two years in patients with relapsing-remitting MS and secondary progressive MS with exacerbations. In patients with primary progressive course of the disease, its activity was established by the progress of the disease within the past year. In patients with the stable course of the disease, there were no exacerbations or worsening of the disease during the specified period of time [4]. In patients with the active course of the disease, there was observed an increase of neurological deficiency by at least 2 points in one of the functional scales (e.g., visual, brainstem, pyramidal, sensory, coordination) or by at least 1 point in two functional systems. Progression of disability could also be seen on the EDSS scale (Expanded Disability Status Scale) [29]: increase by at least 1 point if the initial EDSS score was below 4.0 points, or increase by at least 0.5 points if the initial EDSS score was 4.0 and above [30]. In patients with a progressive course, an increase in neurological deficit by 1 point on the EDSS scale was observed [4]. Neurological examination and EDSS [31] assessment were performed by a certified neurologist with more than 10 years of experience in treating patients with multiple sclerosis. The neurological status of patients according to the EDSS scale is presented in Table 1.

Table 1. EDSS scores of patients with MS

Таблица 1. Показатели баллов по шкале EDSS пациентов с РС

EDSS (functional systems)
Visual	Brainstem	Pyramidal	Cerebellar	Sensory	Bowel and bladder	Cerebral functions	Балл EDSS
EDSS score	0-3 (1)	1-4 (2)	0-4 (2)	0-3 (1)	0-3 (1)	0-2 (1)	1,0-7,0 (3,0)

In 18 patients, an active course of relapsing-remitting MS was observed, in 68 patients, no activity was observed over the past two years, 30 patients had a secondary-progressive course with exacerbations, and the remaining 4 patients had a primary-progressive course of the disease.

The study was conducted using a magnetic resonance imaging (MRI) scanner with a magnetic field strength of 1.5 T. To improve visualization of active demyelination foci, a gadolinium-based contrast agent was administered intravenously.

The MRI protocol for the evaluation of the demyelination foci included the following sequences: T1-weighted images (T1WI) were taken before and after administering of the contrast to identify the active foci and evaluate the accumulation of the contrast; T2-weighted images (T2WI) were used to identify the chronic and new foci of demyelination due to high sensitivity to changes in the tissue structure; FLAIR (Fluid Attenuated Inversion Recovery) was used to attenuate the signal from the fluid and better identification of hyper-intensive foci in the white substance, specifically in the vicinity of the cerebral ventricles; DWI (Diffusion Weighted Imaging) was used to evaluate water diffusion in the tissues and to differentiate the active foci from the chronic ones; PD-weighted images (Proton Density) were used for an additional characterization of demyelinated zones and their differentiation from the normal tissue.

RESULTS AND DISCUSSION

YOLOv8 neural network was used to segment MRI images in patients with MS. YOLOv8 (You Only Look Once, version 8) is the latest iteration of the YOLO family of models designed for real-time object detection and segmentation tasks. This architecture is characterized by high processing speed and prediction accuracy, which makes it suitable for medical image analysis, where not only accuracy is important, but also the efficiency of processing large volumes of data when detecting several classes of objects simultaneously. In our study, foci of demyelination were subcortical, juxtacortical, periventricular and infratentorial localizations.

This study uses the YOLOv8 architecture with modifications intended for improvement of performance on medical data. The model was initially trained on a large set of images (ImageNet) with further training on a specialized dataset consisting of MS patients’ marked-up MRI scans. The images included several MRI sequences (T1-weighted, T2-weighted, FLAIR), which ensured a better representation of lesion morphology.

The architecture of the model includes the following blocks:

Backbone: this block uses a modified version of CSPNet (Cross Stage Partial Network) to extract signs from various levels of abstraction. CSPNet divides the data flow into two parts: one part goes through a sequence of convolutions, whereas the other is sent to later stages. This alleviated the calculation burden and improve the model convergence [32].
Neck: this block joins the signs from various levels using the FPN (Feature Pyramid Network) and the PAN (Path Aggregation Network). FPN assists joining the signs with low and high resolution, and PAN improves transmission of information across the network, which is critical for precise segmentation.
Head: the output layer includes adaptive Anchor Boxes and convolution layers that predict classes of objects, their location and segmentation masks. In this modification of YOLOv8 for the purposes of medical segmentation, the block of deep convolution layer was also used to improve sensitivity to minor details of lesions [33].

To train the model, images sized 512×512 pixels were used. The training was performed using stochastic gradient descent with a momentum of 0.9 and an initial learning rate of 0.001. The dataset was divided into training and testing samples in a 90/30 ratio. Following the segmentation of demyelination foci by the radiologist, images were obtained (Fig. 1) on the basis of which ‘masks’ were formed to train the YOLOv8 algorithm.

Figure 1. A – MRI T2 mode with a focus of demyelination. B – “mask” of demyelination foci obtained as a result of segmentation by a radiologist.

Рисунок 1. А – МРТ Т2 режим с очага демиелинизации. В – «маска» очагов демиелинизации, полученная в результате сегментации врачом-радиологом.

The block diagram of the complete algorithm for creating a training sample, including all stages of primary preparation, their standardization based on the features of differences in the matrix of the image itself, is presented in Figure 2.

Figure 2. Flowchart of the algorithm for creating a training sample.

Рисунок 2. Блок-схема алгоритма создания обучающей выборки.

To evaluate quality of segmentation, Intersection over Union (IoU) and Dice Score metrics were used.

The model was trained using the data from 30 examinations, each having 86 scans on average (total of 2580 scans).

The resulting model is well capable of recognizing demyelination areas on MRI scans with high accuracy and confidence. However, it can be improved to minimize omission of positives. The metrics of the resulting model are shown in Fig. 3.

Figure 3. Metrics characterizing the quality of the model obtained on the test sample. A – function of the F1 graph from the Confidence argument; B – graph of the function Accuracy versus the argument Confidence; C – graph function Confidence from the argument Recall; D – graph function Precision by argument Recall.

Рисунок 3. Метрики, характеризующие качество модели, полученной на тестовой выборке. А – график функции F1 от аргумента Confidence; B – график функции Precision от аргумента Confidence; C – график функции Confidence от аргумента Recall; D – график функции от аргумента Recall.

The summary of metrics characterizing the quality of the resulting model by separate types of demyelination foci is shown in Table 2.

Table 2. Metrics of the resulting model regarding the classification of individual demyelination lesions

Таблица 2. Показатели метрик полученной модели относительно классификации отдельных очагов демиелинизации

Localization of demyelination foci	Precision	Recall	F1-Score	Confidence
Periventricular (1)	0,82	0,76	0,59	0,85
Juxtacortical (2)	0,80	0,71	0,52	0,82
Subcortical (3)	0,78	0,70	0,79	0,80
Infratentorial (4)	0,76	0,68	0,58	0,78
All types of foci	0,79	0,73	0,61	0,81

One of the main indicators of quality of the model’s performance is the confusion matrix shown in Fig. 4. The classifier works with four classes, each matching a certain type of demyelination foci and the class uniting all types of demyelination foci. The model’s prediction are the rows and the truth is the columns. Using the test data, the model predicted 258 areas of which 210 were true, the remaining 48 being false-positives.

Figure 4. Confusion Matrix (0 – all types of demyelination lesions; 1 – periventricular; 2 – juxtacortical; 3 – subcortical; 4 – infratentorial).

Рисунок 4. Матрица ошибок модели (0 – все типы очагов демиелинизации; 1 – перивентрикулярные; 2 – юкстакортикальные; 3 – субкортикальные; 4 – инфратенториальные).

The confusion matrix helps to visualize and understand which classes the model identifies correctly, and which ones cause difficulties. This is critical for further optimization of the model and improvement of its performance.

It is safe to say, therefore, that the evaluation of ‘precision’, ‘recall’, ‘F1-Score’ and ‘confidence’ metrics allows a detailed analysis of quality of the model’s performance and its capability of identifying areas of interest on medical images. Their improvement is necessary to obtain a more stable model to allow using the results of its work in clinical or research practice. These metrics provide a comprehensive evaluation of the model and help in further improvement of its parameters and algorithms, which is critical for the development of an effective clinical decision support system for the diagnosis and treatment of MS.

CONCLUSION

The implementation of the YOLOv8 deep learning algorithm for the automated segmentation of demyelination foci in patients with multiple sclerosis demonstrated high precision and efficiency, which confirms the potential of this approach for clinical practice. Despite the results achieved, the model can be improved in several areas.

Firstly, the values of segmentation completeness (Recall) are still lower than needed, which indicates the possibility of omission of individual foci of demyelination. To improve segmentation completeness, it is possible to look into the usage of hybrid architectures that combine YOLOv8 with other models, e.g. U-Net or nnU-Net that have demonstrated their efficiency in medical segmentation. A combination of advantages of different architectures may improve the model’s capability of identifying even the smallest and hard-to-discern foci. Another prospective line of work could be the implementation of model ensemble when the results provided by several algorithms are joined to improve the overall precision and completeness.

Secondly, one of the ways of further optimization could be the use of data augmentation and synthetic image generation methods. Deep learning models often depend on the volume and diversity of data, and the increase of the data set by generating new MRI scans, especially for complicated or hard-to-reach cases, could improve the precision of segmentation and lower the probability of false positive results. The use of generative adversarial networks (GANs) to create such synthetic data could become an effective approach.

The model can further benefit from using the domain adaptation methods. This approach enables the algorithms rained on one data set to perform efficiently on other sets of data different in quality or scanning methods. In real clinical practice, different MRI machines with different technical parameters are used, which can affect the scan quality. The adaptation of the model to different sets of data, possibly by means of transfer learning, could improve the model’s versatility and performance reliability.

Possible directions for further application of the model include its integration into complex medical decision-making support systems. Automatic segmentation of demyelination foci can be used not only for diagnosis, but also for monitoring disease progression and assessing the effectiveness of therapy. Regular use of MRI with automatic segmentation could assist doctors in following up on the patients’ condition, timely identification of new lesions and evaluation of changes in the existing ones. This will allow for faster adjustment of therapy and prevention of complications.

Another promising direction is the use of the model for training specialists. Automated segmentation algorithms can serve as the basis for educational systems allowing young doctors and radiologists learn by using real medical data and comparing their results with those yielded by the algorithm. This may assist improvement of professional training and lowering the human factor in diagnostics.

Thus, the implementation and further optimization of automated segmentation models, e.g. YOLOv8, may contribute significantly to the quality of multiple sclerosis diagnostics, speed up the data processing and alleviate the burden on the medical professionals.